Configure optional cluster settings

Note

Consider this section as part of the Bootstrap v2 CLI procedure.

During creation of a management cluster using Bootstrap v2, you can configure optional cluster settings using the Container Cloud API by modifying cluster.yaml.template.

To configure optional cluster settings:

  1. Technology Preview. Enable custom host names for cluster machines. When enabled, any machine host name in a particular region matches the related Machine object name. For example, instead of the default kaas-node-<UID>, a machine host name will be master-0. The custom naming format is more convenient and easier to operate with.

    Configuration for custom host names on the management and its future managed clusters

    1. In cluster.yaml.template, find the spec.providerSpec.value.kaas.regional.helmReleases.name: baremetal-provider section.

    2. Under values.config, add customHostnamesEnabled: true:

      regional:
 - helmReleases:
   - name: baremetal-provider
     values:
       config:
         allInOneAllowed: false
         customHostnamesEnabled: true
         internalLoadBalancers: false
   provider: baremetal-provider

    1. In cluster.yaml.template, find the spec.providerSpec.value.kaas.regional section of the required region.

    2. In this section, find baremetal-provider under helmReleases.

    3. Under values.config, add customHostnamesEnabled: true. For example, in region-one:

      regional:
 - helmReleases:
   - name: baremetal-provider
     values:
       config:
         allInOneAllowed: false
         customHostnamesEnabled: true
         internalLoadBalancers: false
   provider: baremetal-provider

  2. Technology Preview. Enable the Linux Audit daemon auditd to monitor activity of cluster processes and prevent potential malicious activity.

    Configuration for auditd

    In cluster.yaml.template, add the auditd parameters:

    spec:
  providerSpec:
    value:
      audit:
        auditd:
          enabled: <bool>
          enabledAtBoot: <bool>
          backlogLimit: <int>
          maxLogFile: <int>
          maxLogFileAction: <string>
          maxLogFileKeep: <int>
          mayHaltSystem: <bool>
          presetRules: <string>
          customRules: <string>
          customRulesX32: <text>
          customRulesX64: <text>

    Configuration parameters for auditd:

    enabled

    Boolean, default - false. Enables the auditd role to install the auditd packages and configure rules. CIS rules: 4.1.1.1, 4.1.1.2.

    enabledAtBoot

    Boolean, default - false. Configures grub to audit processes that can be audited even if they start up prior to auditd startup. CIS rule: 4.1.1.3.

    backlogLimit

    Integer, default - none. Configures the backlog to hold records. If during boot audit=1 is configured, the backlog holds 64 records. If more than 64 records are created during boot, auditd records will be lost with a potential malicious activity being undetected. CIS rule: 4.1.1.4.

    maxLogFile

    Integer, default - none. Configures the maximum size of the audit log file. Once the log reaches the maximum size, it is rotated and a new log file is created. CIS rule: 4.1.2.1.

    maxLogFileAction

    String, default - none. Defines handling of the audit log file reaching the maximum file size. Allowed values:

    • keep_logs - rotate logs but never delete them

    • rotate - add a cron job to compress rotated log files and keep maximum 5 compressed files.

    • compress - compress log files and keep them under the /var/log/auditd/ directory. Requires auditd_max_log_file_keep to be enabled.

    CIS rule: 4.1.2.2.

    maxLogFileKeep

    Integer, default - 5. Defines the number of compressed log files to keep under the /var/log/auditd/ directory. Requires auditd_max_log_file_action=compress. CIS rules - none.

    mayHaltSystem

    Boolean, default - false. Halts the system when the audit logs are full. Applies the following configuration:

    • space_left_action = email

    • action_mail_acct = root

    • admin_space_left_action = halt

    CIS rule: 4.1.2.3.

    customRules

    String, default - none. Base64-encoded content of the 60-custom.rules file for any architecture. CIS rules - none.

    customRulesX32

    String, default - none. Base64-encoded content of the 60-custom.rules file for the i386 architecture. CIS rules - none.

    customRulesX64

    String, default - none. Base64-encoded content of the 60-custom.rules file for the x86_64 architecture. CIS rules - none.

    presetRules

    String, default - none. Comma-separated list of the following built-in preset rules:

    • access

    • actions

    • delete

    • docker

    • identity

    • immutable

    • logins

    • mac-policy

    • modules

    • mounts

    • perm-mod

    • privileged

    • scope

    • session

    • system-locale

    • time-change

    Since Container Cloud 2.28.0 (Cluster releases 17.3.0 and 16.3.0) in the Technology Preview scope, you can collect some of the preset rules indicated above as groups and use them in presetRules:

    • ubuntu-cis-rules - this group contains rules to comply with the Ubuntu CIS Benchmark recommendations, including the following CIS Ubuntu 20.04 v2.0.1 rules:

      • scope - 5.2.3.1

      • actions - same as 5.2.3.2

      • time-change - 5.2.3.4

      • system-locale - 5.2.3.5

      • privileged - 5.2.3.6

      • access - 5.2.3.7

      • identity - 5.2.3.8

      • perm-mod - 5.2.3.9

      • mounts - 5.2.3.10

      • session - 5.2.3.11

      • logins - 5.2.3.12

      • delete - 5.2.3.13

      • mac-policy - 5.2.3.14

      • modules - 5.2.3.19

    • docker-cis-rules - this group contains rules to comply with Docker CIS Benchmark recommendations, including the docker Docker CIS v1.6.0 rules 1.1.3 - 1.1.18.

    You can also use two additional keywords inside presetRules:

    • none - select no built-in rules.

    • all - select all built-in rules. When using this keyword, you can add the ! prefix to a rule name to exclude some rules. You can use the ! prefix for rules only if you add the all keyword as the first rule. Place a rule with the ! prefix only after the all keyword.

    Example configurations:

    • presetRules: none - disable all preset rules

    • presetRules: docker - enable only the docker rules

    • presetRules: access,actions,logins - enable only the access, actions, and logins rules

    • presetRules: ubuntu-cis-rules - enable all rules from the ubuntu-cis-rules group

    • presetRules: docker-cis-rules,actions - enable all rules from the docker-cis-rules group and the actions rule

    • presetRules: all - enable all preset rules

    • presetRules: all,!immutable,!sessions - enable all preset rules except immutable and sessions


    CIS controls
    4.1.3 (time-change)
    4.1.4 (identity)
    4.1.5 (system-locale)
    4.1.6 (mac-policy)
    4.1.7 (logins)
    4.1.8 (session)
    4.1.9 (perm-mod)
    4.1.10 (access)
    4.1.11 (privileged)
    4.1.12 (mounts)
    4.1.13 (delete)
    4.1.14 (scope)
    4.1.15 (actions)
    4.1.16 (modules)
    4.1.17 (immutable)
    Docker CIS controls
    1.1.4
    1.1.8
    1.1.10
    1.1.12
    1.1.13
    1.1.15
    1.1.16
    1.1.17
    1.1.18
    1.2.3
    1.2.4
    1.2.5
    1.2.6
    1.2.7
    1.2.10
    1.2.11

    See also

    Operations Guide: Troubleshooting - The auditd events cause ‘backlog limit exceeded’ messages

  3. Configure OIDC integration:

    LDAP configuration

    Example configuration:

    spec:
  providerSpec:
    value:
      kaas:
        management:
          helmReleases:
          - name: iam
            values:
              keycloak:
                userFederation:
                  providers:
                    - displayName: "<LDAP_NAME>"
                      providerName: "ldap"
                      priority: 1
                      fullSyncPeriod: -1
                      changedSyncPeriod: -1
                      config:
                        pagination: "true"
                        debug: "false"
                        searchScope: "1"
                        connectionPooling: "true"
                        usersDn: "<DN>" # "ou=People, o=<ORGANIZATION>, dc=<DOMAIN_COMPONENT>"
                        userObjectClasses: "inetOrgPerson,organizationalPerson"
                        usernameLDAPAttribute: "uid"
                        rdnLDAPAttribute: "uid"
                        vendor: "ad"
                        editMode: "READ_ONLY"
                        uuidLDAPAttribute: "uid"
                        connectionUrl: "ldap://<LDAP_DNS>"
                        syncRegistrations: "false"
                        authType: "simple"
                        bindCredential: ""
                        bindDn: ""
                  mappers:
                    - name: "username"
                      federationMapperType: "user-attribute-ldap-mapper"
                      federationProviderDisplayName: "<LDAP_NAME>"
                      config:
                        ldap.attribute: "uid"
                        user.model.attribute: "username"
                        is.mandatory.in.ldap: "true"
                        read.only: "true"
                        always.read.value.from.ldap: "false"
                    - name: "full name"
                      federationMapperType: "full-name-ldap-mapper"
                      federationProviderDisplayName: "<LDAP_NAME>"
                      config:
                        ldap.full.name.attribute: "cn"
                        read.only: "true"
                        write.only: "false"
                    - name: "last name"
                      federationMapperType: "user-attribute-ldap-mapper"
                      federationProviderDisplayName: "<LDAP_NAME>"
                      config:
                        ldap.attribute: "sn"
                        user.model.attribute: "lastName"
                        is.mandatory.in.ldap: "true"
                        read.only: "true"
                        always.read.value.from.ldap: "true"
                    - name: "email"
                      federationMapperType: "user-attribute-ldap-mapper"
                      federationProviderDisplayName: "<LDAP_NAME>"
                      config:
                        ldap.attribute: "mail"
                        user.model.attribute: "email"
                        is.mandatory.in.ldap: "false"
                        read.only: "true"
                        always.read.value.from.ldap: "true"

    Note

    • Verify that the userFederation section is located on the same level as the initUsers section.

    • Verify that all attributes set in the mappers section are defined for users in the specified LDAP system. Missing attributes may cause authorization issues.

    For details, see Configure LDAP for IAM.

    Google OAuth configuration

    Example configuration:

    keycloak:
  externalIdP:
    google:
      enabled: true
      config:
        clientId: <Google_OAuth_client_ID>
        clientSecret: <Google_OAuth_client_secret>

    For details, see Configure Google OAuth IdP for IAM.

  4. Disable NTP that is enabled by default. This option disables the management of chrony configuration by Container Cloud to use your own system for chrony management. Otherwise, configure the regional NTP server parameters as described below.

    NTP configuration

    Configure the regional NTP server parameters to be applied to all machines of managed clusters.

    In cluster.yaml.template, add the ntp:servers section with the list of required server names:

    spec:
  ...
  providerSpec:
    value:
      kaas:
      ...
      ntpEnabled: true
        regional:
          - helmReleases:
            - name: <providerName>-provider
              values:
                config:
                  lcm:
                    ...
                    ntp:
                      servers:
                      - 0.pool.ntp.org
                      ...
            provider: <providerName>
            ...

    To disable NTP:

    spec:
  ...
  providerSpec:
    value:
      ...
      ntpEnabled: false
      ...

  5. Applies since Container Cloud 2.26.0 (Cluster release 16.1.0). If you plan to deploy large managed clusters, enable dynamic IP allocation to increase the amount of baremetal hosts to be provisioned in parallel. For details, see Enable dynamic IP allocation.

Now, proceed with completing the bootstrap process using the Container Cloud Bootstrap API as described in Deploy a management cluster using the Container Cloud API.