Set up a bootstrap cluster

The setup of a bootstrap cluster comprises preparation of the seed node, configuration of environment variables, acquisition of the Container Cloud license file, and execution of the bootstrap script. The script eventually generates a link to the Bootstrap web UI for the management cluster deployment.

To set up a bootstrap cluster:

  1. Prepare the seed node:

    Bare metal
    1. Verify that the hardware allocated for the installation meets the minimal requirements described in Requirements for a baremetal-based cluster.

    2. Install basic Ubuntu 20.04 server using standard installation images of the operating system on the bare metal seed node.

    3. Log in to the seed node that is running Ubuntu 20.04.

    4. Prepare the system and network configuration:

      1. Establish a virtual bridge using an IP address of the PXE network on the seed node. Use the following netplan-based configuration file as an example:

        # cat /etc/netplan/config.yaml
        network:
          version: 2
          renderer: networkd
          ethernets:
            ens3:
                dhcp4: false
                dhcp6: false
          bridges:
              br0:
                  addresses:
                  # Replace with IP address from PXE network to create a virtual bridge
                  - 10.0.0.15/24
                  dhcp4: false
                  dhcp6: false
                  # Adjust for your environment
                  gateway4: 10.0.0.1
                  interfaces:
                  # Interface name may be different in your environment
                  - ens3
                  nameservers:
                      addresses:
                      # Adjust for your environment
                      - 8.8.8.8
                  parameters:
                      forward-delay: 4
                      stp: false
        
      2. Apply the new network configuration using netplan:

        sudo netplan apply
        
      3. Verify the new network configuration:

        sudo apt update && sudo apt install -y bridge-utils
        sudo brctl show
        

        Example of system response:

        bridge name     bridge id               STP enabled     interfaces
        br0             8000.fa163e72f146       no              ens3
        

        Verify that the interface connected to the PXE network belongs to the previously configured bridge.

      4. Install the current Docker version available for Ubuntu 20.04:

        sudo apt-get update
        sudo apt-get install docker.io
        
      5. Verify that your logged USER has access to the Docker daemon:

        sudo usermod -aG docker $USER
        
      6. Log out and log in again to the seed node to apply the changes.

      7. Verify that Docker is configured correctly and has access to Container Cloud CDN. For example:

        docker run --rm alpine sh -c "apk add --no-cache curl; \
        curl https://binary.mirantis.com"
        

        The system output must contain a json file with no error messages. In case of errors, follow the steps provided in Troubleshooting.

        Note

        If you require all Internet access to go through a proxy server for security and audit purposes, configure Docker proxy settings as described in the official Docker documentation.

    5. Verify that the seed node has direct access to the Baseboard Management Controller (BMC) of each bare metal host. All target hardware nodes must be in the power off state.

      For example, using the IPMI tool:

      apt install ipmitool
      ipmitool -I lanplus -H 'IPMI IP' -U 'IPMI Login' -P 'IPMI password' \
      chassis power status
      

      Example of system response:

      Chassis Power is off
      
    OpenStack
    1. Verify that the hardware allocated for installation meets minimal requirements described in Requirements for an OpenStack-based cluster.

    2. Configure Docker:

      1. Log in to any personal computer or VM running Ubuntu 20.04 that you will be using as the bootstrap node.

      2. If you use a newly created VM, run:

        sudo apt-get update
        
      3. Install the current Docker version available for Ubuntu 20.04:

        sudo apt install docker.io
        
      4. Grant your USER access to the Docker daemon:

        sudo usermod -aG docker $USER
        
      5. Log off and log in again to the bootstrap node to apply the changes.

      6. Verify that Docker is configured correctly and has access to Container Cloud CDN. For example:

        docker run --rm alpine sh -c "apk add --no-cache curl; \
        curl https://binary.mirantis.com"
        

        The system output must contain no error records. In case of issues, follow the steps provided in Troubleshooting.

    vSphere
    1. Verify that the hardware allocated for installation meets minimal requirements described in Requirements for a VMware vSphere-based cluster.

    2. Configure Ubuntu or RHEL on the bootstrap node:

      • Ubuntu:

        1. Log in to any personal computer or VM running Ubuntu 20.04 that you will be using as the bootstrap node.

        2. If you use a newly created VM, run:

          sudo apt-get update
          
        3. Install the current Docker version available for Ubuntu 20.04:

          sudo apt install docker.io
          
        4. Grant your USER access to the Docker daemon:

          sudo usermod -aG docker $USER
          
        5. Log off and log in again to the bootstrap node to apply the changes.

        6. Verify that Docker is configured correctly and has access to Container Cloud CDN. For example:

          docker run --rm alpine sh -c "apk add --no-cache curl; \
          curl https://binary.mirantis.com"
          

          The system output must contain no error records. In case of issues, follow the steps provided in Troubleshooting.

      • RHEL:

        Note

        RHEL 8.7 is generally available for a bootstrap node since Container Cloud 2.25.0. Before that, it is supported as Technology Preview.

        1. Log in to a VM running RHEL 8.7 that you will be using as a bootstrap node.

        2. Recommended. To avoid the potential Kubernetes upstream issue 3372 causing the cluster re-creation failure, apply the following workaround:

          1. Update the GRUB_CMDLINE_LINUX parameter in the /etc/default/grub file with the systemd.unified_cgroup_hierarchy=1 option.

          2. Update the grub configuration:

            grub2-mkconfig -o /boot/grub2/grub.cfg
            
          3. Reboot the seed node.

        3. If you do not use RedHat Satellite server locally in your infrastructure and require all Internet access to go through a proxy server, including access to RedHat customer portal, configure proxy parameters for subscription-manager using the example below:

          subscription-manager config \
              --server.proxy_scheme=$SCHEME \
              --server.proxy_hostname=$HOST \
              --server.proxy_port=$PORT \
              --server.proxy_user=$USER \
              --server.proxy_password=$PASS \
              --server.no_proxy=$NO_PROXY
          

          Caution

          In MITM proxy deployments, use the internal Red Hat Satellite server to register RHEL machines so that a VM can access this server directly without a MITM proxy.

        4. Attach the RHEL subscription using subscription-manager.

        5. Install the following packages:

          sudo yum install yum-utils wget vim -y
          
        6. Add the Docker mirror according to the operating system major version that is 8 for RHEL 8.7. Provide the proxy URL, if required, or set to _none_.

          sudo cat <<EOF > /etc/yum.repos.d/docker-ee.repo
          [docker-ee]
          name=Docker EE
          gpgcheck=0
          enabled=1
          priority=1
          baseurl=https://repos.mirantis.com/rhel/<RHEL_MAJOR_VERSION>/x86_64/stable-23.0/
          module_hotfixes=1
          proxy=PROXY
          EOF
          
        7. Install and configure Docker:

          sudo yum install docker-ee -y
          sudo systemctl start docker
          sudo chmod 666 /var/run/docker.sock
          
        8. Verify that Docker is configured correctly and has access to Container Cloud CDN. For example:

          docker run --rm alpine sh -c "apk add --no-cache curl; \
          curl https://binary.mirantis.com"
          

          The system output must contain no error records. In case of issues, follow the steps provided in Troubleshooting.

          Note

          If you require all Internet access to go through a proxy server for security and audit purposes, configure Docker proxy settings as described in the official Docker documentation.

    3. Prepare the VMware deployment user setup and permissions.

  2. Prepare the bootstrap script:

    1. Download and run the Container Cloud bootstrap script:

      sudo apt-get update
      sudo apt-get install wget
      wget https://binary.mirantis.com/releases/get_container_cloud.sh
      chmod 0755 get_container_cloud.sh
      ./get_container_cloud.sh
      
    2. Change the directory to the kaas-bootstrap folder created by the script.

  3. Obtain a Container Cloud license file required for the bootstrap:

    1. Select from the following options:

      • Open the email from support@mirantis.com with the subject Mirantis Container Cloud License File or Mirantis OpenStack License File

      • In the Mirantis CloudCare Portal, open the Account or Cloud page

    2. Download the License File and save it as mirantis.lic under the kaas-bootstrap directory on the bootstrap node.

    3. Verify that mirantis.lic contains the previously downloaded Container Cloud license by decoding the license JWT token, for example, using jwt.io.

      Example of a valid decoded Container Cloud license data with the mandatory license field:

      {
          "exp": 1652304773,
          "iat": 1636669973,
          "sub": "demo",
          "license": {
              "dev": false,
              "limits": {
                  "clusters": 10,
                  "workers_per_cluster": 10
              },
              "openstack": null
          }
      }
      

      Warning

      The MKE license does not apply to mirantis.lic. For details about MKE license, see MKE documentation.

  4. For the bare metal provider, export mandatory parameters.

    Bare metal network mandatory parameters

    Export the following mandatory parameters using the commands and table below:

    export KAAS_BM_ENABLED="true"
    #
    export KAAS_BM_PXE_IP="172.16.59.5"
    export KAAS_BM_PXE_MASK="24"
    export KAAS_BM_PXE_BRIDGE="br0"
    
    Bare metal prerequisites data

    Parameter

    Description

    Example value

    KAAS_BM_PXE_IP

    The provisioning IP address in the PXE network. This address will be assigned on the seed node to the interface defined by the KAAS_BM_PXE_BRIDGE parameter described below. The PXE service of the bootstrap cluster uses this address to network boot bare metal hosts.

    172.16.59.5

    KAAS_BM_PXE_MASK

    The PXE network address prefix length to be used with the KAAS_BM_PXE_IP address when assigning it to the seed node interface.

    24

    KAAS_BM_PXE_BRIDGE

    The PXE network bridge name that must match the name of the bridge created on the seed node during the Set up a bootstrap cluster stage.

    br0

  5. Optional. Add the following environment variables to bootstrap the cluster using proxy:

    • HTTP_PROXY

    • HTTPS_PROXY

    • NO_PROXY

    • PROXY_CA_CERTIFICATE_PATH

    Example snippet:

    export HTTP_PROXY=http://proxy.example.com:3128
    export HTTPS_PROXY=http://user:pass@proxy.example.com:3128
    export NO_PROXY=172.18.10.0,registry.internal.lan
    export PROXY_CA_CERTIFICATE_PATH="/home/ubuntu/.mitmproxy/mitmproxy-ca-cert.cer"
    

    The following formats of variables are accepted:

    Proxy configuration data

    Variable

    Format

    HTTP_PROXY
    HTTPS_PROXY
    • http://proxy.example.com:port - for anonymous access.

    • http://user:password@proxy.example.com:port - for restricted access.

    NO_PROXY

    Comma-separated list of IP addresses or domain names. For the vSphere provider, mandatory to add host[:port] of the vCenter server.

    PROXY_CA_CERTIFICATE_PATH

    Optional. Absolute path to the proxy CA certificate for man-in-the-middle (MITM) proxies. Must be placed on the bootstrap node to be trusted. For details, see Install a CA certificate for a MITM proxy on a bootstrap node.

    Warning

    If you require Internet access to go through a MITM proxy, ensure that the proxy has streaming enabled as described in Enable streaming for MITM.

    For proxy implementation details, see Proxy and cache support.

    After the bootstrap cluster is set up, the bootstrap-proxy object is created with the provided proxy settings. You can use this object later for the Cluster object configuration.

  6. Deploy the bootstrap cluster:

    ./bootstrap.sh bootstrapv2
    

    When the bootstrap is complete, the system outputs a link to the Bootstrap web UI.

  7. Make sure that port 80 is open for localhost to prevent security requirements for the seed node:

    Note

    Kind uses port mapping for the master node.

    telnet localhost 80
    

    Example of a positive system response:

    Connected to localhost.
    

    Example of a negative system response:

    telnet: connect to address ::1: Connection refused
    telnet: Unable to connect to remote host
    

    To open port 80:

    iptables -A INPUT -p tcp --dport 80 -j ACCEPT
    
  8. Access the Bootstrap web UI. It does not require any authorization.

    The bootstrap cluster setup automatically creates the following objects that you can view in the Bootstrap web UI:

    • Bootstrap SSH key

      The SSH key pair is automatically generated by the bootstrap script and the private key is added to the kaas-bootstrap folder. The public key is automatically created in the bootstrap cluster as the bootstrap-key object. It will be used later for setting up the cluster machines.

    • Bootstrap proxy

      If a bootstrap cluster is configured with proxy settings, the bootstrap-proxy object is created. It will be automatically used in the cluster configuration unless a custom proxy is specified.

    • Management kubeconfig

      If a bootstrap cluster is provided with the management cluster kubeconfig, it will be uploaded as a secret to the bootstrap cluster to the default and kaas projects as management-kubeconfig.