Configuration options

auth table

Parameter

Required

Description

backend

no

The name of the authorization back end to use, managed or ldap.

Default: managed

default_new_user_role

no

The role assigned to new users for their private resource sets.

Valid values: admin, viewonly, scheduler, restrictedcontrol, or fullcontrol.

Default: restrictedcontrol

auth.sessions

Parameter

Required

Description

lifetime_minutes

no

The initial session lifetime, in minutes.

Default: 60

renewal_threshold_minutes

no

The length of time, in minutes, before the expiration of a session where, if used, a session will be extended by the current configured lifetime from then. A value of 0 disables session extension.

Default: 20

per_user_limit

no

The maximum number of sessions that a user can have simultaneously active. If creating a new session will put a user over this limit, the least recently used session is deleted.

A value of 0 disables session limiting.

Default: 10

store_token_per_session

no

If set, the user token is stored in sessionStorage instead of localStorage. Setting this option logs the user out and requires that they log back in, as they are actively changing the manner in which their authentication is stored.

auth.external_identity_provider (optional)

Configures MKE with an external OpenID Connect (OIDC) identity provider.

Parameter

Required

Description

wellKnownConfigUrl

yes

Sets the OpenID discovery endpoint, ending in .well-known/openid-configuration, for your identity provider.

clientID

yes

Sets the client ID, which you obtain from your identity provider.

clientSecret

no (recommended)

Sets the client secret, which you obtain from your identity provider.

usernameClaim

no

Sets the unique JWT ID token claim that contains the user names from your identity provider.

Default: sub

caBundle

no

Sets the PEM certificate bundle that MKE uses to authenticate the discovery, issuer, and JWKs endpoints.

httpProxy

no

Sets the HTTP proxy for your identity provider.

httpsProxy

no

Sets the HTTPS proxy for your identity provider.

issuer

no

Sets the ID token issuer. If left blank, the value is obtained automatically from the discovery endpoint.

userServiceId

no

Sets the MKE service ID with the JWK URI for the identity provider. If left blank, the service ID is generated automatically.

Warning

Do not remove or replace an existing value.

auth.external_identity_provider.signInCriteria array (optional)

An array of claims that ID tokens require for use with MKE.

Parameter

Required

Description

term

yes

Sets the name of the claim.

value

yes

Sets the value for the claim in the form of a string.

matchType

yes

Sets how MKE evaluates the JWT claim.

Valid values:

  • must - the JWT claim value must be the same as the configuration value.

  • contains - the JWT claim value must contain the configuration value.

auth.external_identity_provider.adminRoleCriteria array (optional)

An array of claims that admin user ID tokens require for use with MKE. Creating a new account using a token that satisfies the criteria determined by this array automatically produces an administrator account.

Parameter

Required

Description

term

yes

Sets the name of the claim.

value

yes

Sets the value for the claim in the form of a string.

matchType

yes

Sets how the JWT claim is evaluated.

Valid values:

  • must - the JWT claim value must be the same as the configuration value.

  • contains - the JWT claim value must contain the configuration value.

auth.account_lock (optional)

Available since MKE 3.5.3

Parameter

Required

Description

enabled

no

Sets whether the MKE account lockout feature is enabled.

failureTrigger

no

Sets the number of failed log in attempts that can occur before an account is locked.

durationSeconds

no

Sets the desired lockout duration in seconds. A value of 0 indicates that the account will remain locked until it is unlocked by an administrator.

hardening_configuration (optional)

The hardening_enabled option must be set to true to enable all other hardening_configuration options.

Parameter

Required

Description

hardening_enabled

no

Parent option that when set to true enables security hardening configuration options: limit_kernel_capabilities, pid_limit, and pid_limit_unspecified.

Default: false

limit_kernel_capabilities

no

The option can only be enabled when hardening_enabled is set to true.

Limits kernel capabilities to the minimum required by each container.

Components run using Docker default capabilities by default. When you enable limit_kernel_capabilities all capabilities are dropped, except those that are specifically in use by the component. Several components run as privileged, with capabilities that cannot be disabled.

Default: false

pid_limit

no

The option can only be enabled when hardening_enabled is set to true.

Sets the maximum number of PIDs MKE can allow for their respective orchestrators.

The pid_limit option must be set to the default 0 when it is not in use.

Default: 0

pid_limit_unspecified

no

The option can only be enabled when hardening_enabled is set to true.

When set to false, enables PID limiting, using the pid_limit option value for the associated orchestrator.

Default: true

registries array (optional)

An array of tables that specifies the MSR instances that are managed by the current MKE instance.

Parameter

Required

Description

host_address

yes

Sets the address for connecting to the MSR instance tied to the MKE cluster.

service_id

yes

Sets the MSR instance’s OpenID Connect Client ID, as registered with the Docker authentication provider.

ca_bundle

no

Specifies the root CA bundle for the MSR instance if you are using a custom certificate authority (CA). The value is a string with the contents of a ca.pem file.

audit_log_configuration table (optional)

Configures audit logging options for MKE components.

Parameter

Required

Description

level

no

Specifies the audit logging level.

Valid values: empty (to disable audit logs), metadata, request.

Default: empty

support_dump_include_audit_logs

no

Sets support dumps to include audit logs in the logs of the ucp-controller container of each manager node.

Valid values: true, false.

Default: false

scheduling_configuration table (optional)

Specifies scheduling options and the default orchestrator for new nodes.

Note

If you run a kubectl command, such as kubectl describe nodes, to view scheduling rules on Kubernetes nodes, the results that present do not reflect the MKE admin settings conifguration. MKE uses taints to control container scheduling on nodes and is thus unrelated to the kubectl Unschedulable boolean flag.

Parameter

Required

Description

enable_admin_ucp_scheduling

no

Determines whether administrators can schedule containers on manager nodes.

Valid values: true, false.

Default: false

You can also set the parameter using the MKE web UI:

  1. Log in to the MKE web UI as an administrator.

  2. Click the user name drop-down in the left-side navigation panel.

  3. Click Admin Settings > Orchestration to view the Orchestration screen.

  4. Scroll down to the Container Scheduling section and toggle on the Allow administrators to deploy containers on MKE managers or nodes running MSR slider.

default_node_orchestrator

no

Sets the type of orchestrator to use for new nodes that join the cluster.

Valid values: swarm, kubernetes.

Default: swarm

tracking_configuration table (optional)

Specifies the analytics data that MKE collects.

Parameter

Required

Description

disable_usageinfo

no

Set to disable analytics of usage information.

Valid values: true, false.

Default: false

disable_tracking

no

Set to disable analytics of API call information.

Valid values: true, false.

Default: false

cluster_label

no

Set a label to be included with analytics.

ops_care

no

Set to enable OpsCare.

Valid values: true, false.

Default: false

trust_configuration table (optional)

Specifies whether MSR images require signing.

Parameter

Required

Description

require_content_trust

no

Set to require the signing of images by content trust.

Valid values: true, false.

Default: false

You can also set the parameter using the MKE web UI:

  1. Log in to the MKE web UI as an administrator.

  2. Click the user name drop-down in the left-side navigation panel.

  3. Click Admin Settings > Docker Content Trust to open the Content Trust Settings screen.

  4. Toggle on the Run only signed images slider.

require_signature_from

no

A string array that specifies which users or teams must sign images.

allow_repos

no

A string array that specifies repos that are to bypass content trust check, for example, ["docker.io/mirantis/dtr-rethink" , "docker.io/mirantis/dtr-registry" ....].

log_configuration table (optional)

Configures the logging options for MKE components.

Parameter

Required

Description

protocol

no

The protocol to use for remote logging.

Valid values: tcp, udp.

Default: tcp

host

no

Specifies a remote syslog server to receive sent MKE controller logs. If omitted, controller logs are sent through the default Docker daemon logging driver from the ucp-controller container.

level

no

The logging level for MKE components.

Valid values (syslog priority levels): debug, info, notice, warning, err, crit, alert, emerg.

license_configuration table (optional)

Enables automatic renewal of the MKE license.

Parameter

Required

Description

auto_refresh

no

Set to enable attempted automatic license renewal when the license nears expiration. If disabled, you must manually upload renewed license after expiration.

Valid values: true, false.

Default: true

custom headers (optional)

Included when you need to set custom API headers. You can repeat this section multiple times to specify multiple separate headers. If you include custom headers, you must specify both name and value.

[[custom_api_server_headers]]

Item

Description

name

Set to specify the name of the custom header with name = “X-Custom-Header-Name”.

value

Set to specify the value of the custom header with value = “Custom Header Value”.

user_workload_defaults (optional)

A map describing default values to set on Swarm services at creation time if those fields are not explicitly set in the service spec.

[user_workload_defaults]

[user_workload_defaults.swarm_defaults]

Parameter

Required

Description

[tasktemplate.restartpolicy.delay]

no

Delay between restart attempts. The value is input in the <number><value type> formation. Valid value types include:

  • ns = nanoseconds

  • us = microseconds

  • ms = milliseconds

  • s = seconds

  • m = minutes

  • h = hours

Default: value = "5s"

[tasktemplate.restartpolicy.maxattempts]

no

Maximum number of restarts before giving up.

Default: value = "3"

cluster_config table (required)

Configures the cluster that the current MKE instance manages.

The dns, dns_opt, and dns_search settings configure the DNS settings for MKE components. These values, when assigned, override the settings in a container /etc/resolv.conf file.

Parameter

Required

Description

controller_port

yes

Sets the port that the ucp-controller monitors.

Default: 443

kube_apiserver_port

yes

Sets the port the Kubernetes API server monitors.

kube_protect_kernel_defaults

no

Protects kernel parameters from being overridden by kubelet.

Default: false.

Important

When enabled, kubelet can fail to start if the following kernel parameters are not properly set on the nodes before you install MKE or before adding a new node to an existing cluster:

vm.panic_on_oom=0
vm.overcommit_memory=1
kernel.panic=10
kernel.panic_on_oops=1
kernel.keys.root_maxkeys=1000000
kernel.keys.root_maxbytes=25000000

For more information, refer to Configure kernel parameters.

kube_api_server_auditing

no

Enables auditing to the log file in the kube-apiserver container.

Important

  • Prior to using kube_api_server_auditing you must first enable auditing in MKE. Refer to Enable MKE audit logging for detailed information.

  • Before you enable the kube_api_server_auditing option, verify that it does not conflict with MKE options that are already set.

For more information, refer to the official Kubernetes documentation Troubleshooting Clusters - Audit backends.

Default: false.

swarm_port

yes

Sets the port that the ucp-swarm-manager monitors.

Default: 2376

swarm_strategy

no

Sets placement strategy for container scheduling. Be aware that this does not affect swarm-mode services.

Valid values: spread, binpack, random.

dns

yes

Array of IP addresses that serve as nameservers.

dns_opt

yes

Array of options in use by DNS resolvers.

dns_search

yes

Array of domain names to search whenever a bare unqualified host name is used inside of a container.

profiling_enabled

no

Determines whether specialized debugging endpoints are enabled for profiling MKE performance.

Valid values: true, false.

Default: false

authz_cache_timeout

no

Sets the timeout in seconds for the RBAC information cache of MKE non-Kubernetes resource listing APIs. Setting changes take immediate effect and do not require a restart of the MKE controller.

Default: 0 (cache is not enabled)

Once you enable the cache, the result of non-Kubernetes resource listing APIs only reflects the latest RBAC changes for the user when the cached RBAC info times out.

kv_timeout

no

Sets the key-value store timeout setting, in milliseconds.

Default: 5000

kv_snapshot_count

Required

Sets the key-value store snapshot count.

Default: 20000

external_service_lb

no

Specifies an optional external load balancer for default links to services with exposed ports in the MKE web interface.

cni_installer_url

no

Specifies the URL of a Kubernetes YAML file to use to install a CNI plugin. Only applicable during initial installation. If left empty, the default CNI plugin is put to use.

metrics_retention_time

no

Sets the metrics retention time.

metrics_scrape_interval

no

Sets the interval for how frequently managers gather metrics from nodes in the cluster.

metrics_disk_usage_interval

no

Sets the interval for the gathering of storage metrics, an operation that can become expensive when large volumes are present.

nvidia_device_plugin

no

Enables the nvidia-gpu-device-plugin, which is disabled by default.

rethinkdb_cache_size

no

Sets the size of the cache for MKE RethinkDB servers.

Default: 1GB

Leaving the field empty or specifying auto instructs RethinkDB to automatically determine the cache size.

exclude_server_identity_headers

no

Determines whether the X-Server-Ip and X-Server-Name headers are disabled.

Valid values: true, false.

Default: false

cloud_provider

no

Sets the cloud provider for the Kubernetes cluster.

pod_cidr

yes

Sets the subnet pool from which the IP for the Pod should be allocated from the CNI IPAM plugin.

Default: 192.168.0.0/16

calico_mtu

no

Sets the maximum transmission unit (MTU) size for the Calico plugin.

ipip_mtu

no

Sets the IPIP MTU size for the Calico IPIP tunnel interface.

azure_ip_count

yes

Sets the IP count for Azure allocator to allocate IPs per Azure virtual machine.

service_cluster_ip_range

yes

Sets the subnet pool from which the IP for Services should be allocated.

Default: 10.96.0.0/16

nodeport_range

yes

Sets the port range for Kubernetes services within which the type NodePort can be exposed.

Default: 32768-35535

custom_kube_api_server_flags

no

Sets the configuration options for the Kubernetes API server.

Be aware that this parameter function is only for development and testing. Arbitrary Kubernetes configuration parameters are not tested and supported under the MKE Software Support Agreement.

custom_kube_controller_manager_flags

no

Sets the configuration options for the Kubernetes controller manager.

Be aware that this parameter function is only for development and testing. Arbitrary Kubernetes configuration parameters are not tested and supported under the MKE Software Support Agreement.

custom_kubelet_flags

no

Sets the configuration options for kubelet.

Be aware that this parameter function is only for development and testing. Arbitrary Kubernetes configuration parameters are not tested and supported under the MKE Software Support Agreement.

custom_kube_scheduler_flags

no

Sets the configuration options for the Kubernetes scheduler.

Be aware that this arameter function is only for development and testing. Arbitrary Kubernetes configuration parameters are not tested and supported under the MKE Software Support Agreement.

local_volume_collection_mapping

no

Set to store data about collections for volumes in the MKE local KV store instead of on the volume labels. The parameter is used to enforce access control on volumes.

manager_kube_reserved_resources

no

Reserves resources for MKE and Kubernetes components that are running on manager nodes.

worker_kube_reserved_resources

no

Reserves resources for MKE and Kubernetes components that are running on worker nodes.

kubelet_max_pods

yes

Sets the number of Pods that can run on a node.

Maximum: 250

Default: 110

kubelet_pods_per_core

no

Sets the maximum number of Pods per core.

0 indicates that there is no limit on the number of Pods per core. The number cannot exceed the kubelet_max_pods setting.

Recommended: 10

Default: 0

secure_overlay

no

Enables IPSec network encryption in Kubernetes.

Valid values: true, false.

Default: false

image_scan_aggregation_enabled

no

Enables image scan result aggregation. The feature displays image vulnerabilities in shared resource/containers and shared resources/images pages.

Valid values: true, false.

Default: false

swarm_polling_disabled

no

Determines whether resource polling is disabled for both Swarm and Kubernetes resources, which is recommended for production instances.

Valid values: true, false.

Default: false

oidc_client_id

no

Sets the OIDC client ID, using the eNZi service ID that is in the ODIC authorization flow.

hide_swarm_ui

no

Determines whether the UI is hidden for all Swarm-only object types (has no effect on Admin Settings).

Valid values: true, false.

Default: false

You can also set the parameter using the MKE web UI:

  1. Log in to the MKE web UI as an administrator.

  2. In the left-side navigation panel, click the user name drop-down.

  3. Click Admin Settings > Tuning to open the Tuning screen.

  4. Toggle on the Hide Swarm Navigation slider located under the Configure MKE UI heading.

unmanaged_cni

yes

Sets Calico as the CNI provider, managed by MKE. Note that Calico is the default CNI provider.

calico_ebpf_enabled

yes

Enables Calico eBPF mode.

kube_default_drop_masq_bits

yes

Sets the use of Kubernetes default values for iptables drop and masquerade bits.

kube_proxy_mode

yes

Sets the operational mode for kube-proxy.

Valid values: iptables, ipvs, disabled.

Default: iptables

cipher_suites_for_kube_api_server

no

Sets the value for the kube-apiserver --tls-cipher-suites parameter.

cipher_suites_for_kubelet

no

Sets the value for the kubelet --tls-cipher-suites parameter.

cipher_suites_for_etcd_server

no

Sets the value for the etcd server --cipher-suites parameter.

image_prune_schedule Available since MKE 3.5.3

no

Sets the cron expression used for the scheduling of image pruning. The parameter accepts either full crontab specifications or descriptors, but not both.

  • Full crontab specifications, which include <seconds> <minutes> <hours> <day of month> <month> <day of week>. For example, "0 0 0 * * *".

  • Descriptors, which are textual in nature, with a preceding @ symbol. For example: "@midnight" or "@every 1h30m".

Refer to the cron documentation for more information.

cpu_usage_banner_threshold

no

Sets the CPU usage threshold, above which the MKE web UI displays a warning banner.

Default: 20.

cpu_usage_banner_scrape_interval

no

Sets the MKE CPU usage measurement interval, which enables the function of the cpu_usage_banner_threshold option.

Default: "10m".

etcd_storage_quota Available since MKE 3.5.5

no

Sets the etcd storage size limit.

Example values: 500M, 4GB, 8G.

Default value: 2G.

nvidia_device_partitioner Available since MKE 3.5.6

no

Enables the NVIDIA device partitioner.

Default: true.

kube_api_server_profiling_enabled

no

Enables profiling for the Kubernetes API server.

Default: true.

kube_controller_manager_profiling_enabled

no

Enables profiling for the Kubernetes controller manager.

Default: true.

kube_scheduler_profiling_enabled

no

Enables profiling for the Kubernetes scheduler.

Default: true.

kube_scheduler_bind_to_all

no

Enables kube scheduler to bind to all available network interfaces, rather than just localhost.

Default: false.

use_flex_volume_driver

no

Extends support of FlexVolume drivers, which have been deprecated since the release of MKE 3.4.13.

Default: false.

pubkey_auth_cache_enabled

no

Warning

Implement pubkey_auth_cache_enabled only in cases in which there are certain performance issues in high-load clusters, and only under the guidance of Mirantis Support personnel.

Enables public key authentication cache.

Note

ucp-controller must be restarted for setting changes to take effect.

Default: false.

shared_sans

no

Subject alternative names for manager nodes.

cluster_config.image_prune_whitelist (optional)

Available since MKE 3.5.3

Configures the images that you do not want removed by MKE image pruning.

Note

Where possible, use the image ID to specify the image rather than the image name.

Parameter

Required

Description

key

yes

Sets the filter key.

Valid values: dangling, label, before, since, and reference.

For more information, refer to the Docker documentation on Filtering.

value

yes

Sets the filter value.

For more information, refer to the Docker documentation on Filtering.

cluster_config.ingress_controller (optional)

Set the configuration for the NGINX Ingress Controller to manage traffic that originates outside of your cluster (ingress traffic).

Note

Prior versions of MKE use Istio Ingress to manage traffic that originates from outside of the cluster, which employs many of the same parameters as NGINX Ingress Controller.

Parameter

Required

Description

enabled

No

Disables HTTP ingress for Kubernetes.

Valid values: true, false.

Default: false

ingress_num_replicas

No

Sets the number of NGINX Ingress Controller deployment replicas.

Default: 2

ingress_external_ips

No

Sets the list of external IPs for Ingress service.

Default: [] (empty)

ingress_enable_lb

No

Enables an external load balancer.

Valid values: true, false.

Default: false

ingress_preserve_client_ip

No

Enables preserving inbound traffic source IP.

Valid values: true, false.

Default: false

ingress_exposed_ports

No

Sets ports to expose.

For each port, provide arrays that contain the following port information (defaults as displayed):

  • name = http2

  • port = 80

  • target_port = 0

  • node_port = 33000


  • name = https

  • port = 443

  • target_port = 0

  • node_port = 33001


  • name = tcp

  • port = 31400

  • target_port = 0

  • node_port = 33002

ingress_node_affinity

No

Sets node affinity.

  • key = com.docker.ucp.manager

  • value = ""

  • target_port = 0

  • node_port = 0

ingress_node_toleration

No

Sets node toleration.

For each node, provide an array that contains the following information (defaults as displayed):

  • key = com.docker.ucp.manager

  • value = ""

  • operator = Exists

  • effect = NoSchedule

config_map

No

Sets advanced options for the NGINX proxy.

NGINX Ingress Controller uses ConfigMap to configure the NGINX proxy. For the complete list of available options, refer to the NGINX Ingress Controller documentation ConfigMap: configuration options.

Examples:

  • map-hash-bucket-size = "128"

  • ssl-protocols = "SSLv2"

iSCSI (optional)

Configures iSCSI options for MKE.

Parameter

Required

Description

--storage-iscsi=true

no

Enables iSCSI-based Persistent Volumes in Kubernetes.

Valid values: true, false.

Default: false

--iscsiadm-path=<path>

no

Specifies the path of the iscsiadm binary on the host.

Default: /usr/sbin/iscsiadm

--iscsidb-path=<path>

no

Specifies the path of the iscsi database on the host.

Default: /etc/iscsi

pre_logon_message

Configures a pre-logon message.

Parameter

Required

Description

pre_logon_message

no

Sets a pre-logon message to alert users prior to log in.

backup_schedule_config (optional)

Configures backup scheduling and notifications for MKE.

Parameter

Required

Description

notification-delay

yes

Sets the number of days that elapse before a user is notified that they have not performed a recent backup. Set to -1 to disable notifications.

Default: 7

enabled

yes

Enables backup scheduling.

Valid values: true, false.

Default: false

path

yes

Sets the storage path for scheduled backups. Use chmod o+w /<path> to ensure that other users have write privileges.

no_passphrase

yes

Sets whether a passphrase is necessary to encrypt the TAR file. A value of true negates the use of a passphrase. A non-empty value in the passphrase parameter requires that no-passphrase be set to false.

Default: false

passphrase

yes

Encrypts the TAR file with a passphrase for all scheduled backups. Must remain empty if no_passphrase is set to true.

Do not share the configuration file if a passphrase is used, as the passphrase displays in plain text.

cron_spec

yes

Sets the cron expression in use for scheduling backups. The parameter accepts either full crontab specifications or descriptors, but not both.

  • Full crontab specifications include <seconds> <minutes> <hours> <day of month> <month> <day of week>. For example: "0 0 0 * * *".

  • Descriptors, which are textual in nature, have a preceding @ symbol. For example: "@midnight" or "@every 1h30m".

For more information, refer to the cron documentation.

include_logs

yes

Determines whether a log file is generated in addition to the backup. Refer to backup for more information.

backup_limits

yes

Sets the number of backups to store. Once this number is reached, older backups are deleted. Set to -1 to disable backup rotation.