Prepare the virtual machine template

To deploy Mirantis Container Cloud on the vSphere-based environment, prepare the virtual machine (VM) template for cluster machines that fits the following requirements:

  1. The VMware Tools package is installed.

  2. The cloud-init utility is installed and configured with the specific VMwareGuestInfo data source.

  3. For RHEL deployments, the virt-who service is enabled and configured to connect to the VMware vCenter Server to properly apply the RHEL subscriptions on the nodes. The virt-who service can run on a standalone machine or can be integrated into a VM template.

The following procedures describe how to meet the requirements above either using the Container Cloud script or manually.

To prepare the VM template using the Container Cloud script:

  1. Prepare the Container Cloud bootstrap and modify templates/vsphere/vsphere-config.yaml.template and templates/vsphere/cluster.yaml.template as described in Bootstrap a management cluster, steps 1-9.

  2. Download the ISO image depending on the target OS:

  3. Export the following variables:

    • The path to the downloaded ISO file.

    • The vSphere cluster name.

    • The OS name: rhel, ubuntu, or centos.

    • The OS version: 7.8, 7.9, or 8.4 TechPreview for RHEL; 7.9 for CentOS TechPreview, 20.04 for Ubuntu.

    • Optional. The virt-who user name and password for RHEL deployments.

    For example, for RHEL:

    export KAAS_VSPHERE_ENABLED=true
export VSPHERE_RO_USER=virt-who-user
export VSPHERE_RO_PASSWORD=virt-who-user-password
export VSPHERE_PACKER_ISO_FILE=$(pwd)/iso-file.dvd.iso
export VSPHERE_CLUSTER_NAME=vsphere-cluster-name
export VSPHERE_PACKER_IMAGE_OS_NAME=rhel
export VSPHERE_PACKER_IMAGE_OS_VERSION=7.9
    Optional variables

    Variable

    Description

    VSPHERE_VM_TIMEZONE

    Time zone for virtual machines. Defaults to America/New_York.

  4. Optional. If you require all Internet access to go through a proxy server, in bootstrap.env, add the following environment variables:

    • HTTP_PROXY

    • HTTPS_PROXY

    • NO_PROXY

    • PROXY_CA_CERTIFICATE_PATH

    Example snippet:

    export HTTP_PROXY=http://proxy.example.com:3128
export HTTPS_PROXY=http://user:pass@proxy.example.com:3128
export NO_PROXY=172.18.10.0,registry.internal.lan
export PROXY_CA_CERTIFICATE_PATH="/home/ubuntu/.mitmproxy/mitmproxy-ca-cert.cer"

    The following formats of variables are accepted:

    Proxy configuration data

    Variable

    Format
    HTTP_PROXY
    HTTPS_PROXY

    • http://proxy.example.com:port - for anonymous access.

    • http://user:password@proxy.example.com:port - for restricted access.

    NO_PROXY

    Comma-separated list of IP addresses or domain names. Mandatory to add host[:port] of the vCenter server.

    PROXY_CA_CERTIFICATE_PATH

    Optional. Path to the proxy CA certificate for man-in-the-middle (MITM) proxies. Must be placed on the bootstrap node to be trusted. For details, see Install a CA certificate for a MITM proxy on a bootstrap node.

    Warning

    If you require Internet access to go through a MITM proxy, ensure that the proxy has streaming enabled as described in Enable streaming for MITM.

    Note

    • This parameter is generally available for the OpenStack, bare metal, Equinix Metal with private networking, AWS, and vSphere providers.

    • For MOSK-based deployments, the parameter is generally available since MOSK 22.4.

    • For Azure and Equinix Metal with public networking, the feature is not supported.

    For implementation details, see Proxy and cache support.

    Caution

    In MITM proxy deployments, use the internal Red Hat Satellite server to register RHEL machines so that a VM can access this server directly without a MITM proxy.

    For the list of Mirantis resources and IP addresses to be accessible from the Container Cloud clusters, see Requirements for a VMware vSphere-based cluster.

  5. Prepare the VM template:

    ./bootstrap.sh vsphere_template

  6. After the template is prepared, set the <vSphereVMTemplatePath> parameter in templates/vsphere/machines.yaml.template as described in Bootstrap a management cluster.

To prepare the VM template manually:

  1. Run a VM on the vSphere data center with the DVD ISO mounted to it. Specify the amount of resources that will be used in the Container Cloud setup. A minimal resources configuration must match the Requirements for a VMware vSphere-based cluster for a vSphere-based Container Cloud cluster.

    You can create a VM with the disk size larger than the default value of 120 GB. If so, set the selected disk size in machines.yaml.template during cluster templates configuration before bootstrap.

  2. Bootstrap the OS using vSphere Web Console. Select a minimal setup in the VM installation configuration. Create a user with root or sudo permissions to access the machine.

  3. Log in to the VM when it starts.

  4. Optional. If you require all Internet access to go through a proxy server, in bootstrap.env, add the following environment variables:

    • HTTP_PROXY

    • HTTPS_PROXY

    • NO_PROXY

    • PROXY_CA_CERTIFICATE_PATH

    Example snippet:

    export HTTP_PROXY=http://proxy.example.com:3128
export HTTPS_PROXY=http://user:pass@proxy.example.com:3128
export NO_PROXY=172.18.10.0,registry.internal.lan
export PROXY_CA_CERTIFICATE_PATH="/home/ubuntu/.mitmproxy/mitmproxy-ca-cert.cer"

    The following formats of variables are accepted:

    Proxy configuration data

    Variable

    Format
    HTTP_PROXY
    HTTPS_PROXY

    • http://proxy.example.com:port - for anonymous access.

    • http://user:password@proxy.example.com:port - for restricted access.

    NO_PROXY

    Comma-separated list of IP addresses or domain names. Mandatory to add host[:port] of the vCenter server.

    PROXY_CA_CERTIFICATE_PATH

    Optional. Path to the proxy CA certificate for man-in-the-middle (MITM) proxies. Must be placed on the bootstrap node to be trusted. For details, see Install a CA certificate for a MITM proxy on a bootstrap node.

    Warning

    If you require Internet access to go through a MITM proxy, ensure that the proxy has streaming enabled as described in Enable streaming for MITM.

    Note

    • This parameter is generally available for the OpenStack, bare metal, Equinix Metal with private networking, AWS, and vSphere providers.

    • For MOSK-based deployments, the parameter is generally available since MOSK 22.4.

    • For Azure and Equinix Metal with public networking, the feature is not supported.

    For implementation details, see Proxy and cache support.

    Caution

    In MITM proxy deployments, use the internal Red Hat Satellite server to register RHEL machines so that a VM can access this server directly without a MITM proxy.

    For the list of Mirantis resources and IP addresses to be accessible from the Container Cloud clusters, see Requirements for a VMware vSphere-based cluster.

  5. For RHEL, attach your RHEL license for Virtual Datacenters to the VM:

    subscription-manager register
# automatic subscription selection:
subscription-manager attach --auto
# or specify pool id:
subscription-manager attach --pool=<POOL_ID>
# verify subscription status
subscription-manager status

  6. Select from the following options:

    • Prepare the operating system automatically:

      1. Download the automation script:

        curl https://gerrit.mcp.mirantis.com/plugins/gitiles/kubernetes/vmware-guestinfo/+/refs/tags/v1.1.5/install.sh?format=TEXT | \
base64 -d > install.sh
chmod +x install.sh

      2. Optional. For RHEL, export the vCenter Server credentials of the read-only user. For example:

        export VC_SERVER='vcenter1.example.com'
export VC_USER='domain\vmware_read_only_username'
export VC_PASSWORD='password!23'
# optional parameters:
export VC_HYPERVISOR_ID=hostname
export VC_FILTER_HOSTS="esx1.example.com, esx2.example.com"
export VCENTER_CONFIG_PATH="/etc/virt-who.d/vcenter.conf"

      3. Run the installation script:

        ./install.sh

    • Prepare the operating system manually:

      1. Install the open-vm-tools package version 11.0.5 or later with dependencies and verify its version:

        yum install open-vm-tools net-tools perl -y
vmtoolsd --version
vmware-toolbox-cmd --version

      2. Install and configure cloud-init:

        1. Install the cloud-init package and verify its version:

          • 19.4 or later for RHEL 7.8 and 7.9, or CentOS 7.9 TechPreview

          • 20.3 or later for RHEL 8.4 TechPreview

          • 22.1 or later for Ubuntu 20.04

          yum install cloud-init -y
cloud-init --version

        2. For RHEL and CentOS:

          1. Download the VMwareGuestInfo data source files:

            curl https://gerrit.mcp.mirantis.com/plugins/gitiles/kubernetes/vmware-guestinfo/+/refs/tags/v1.1.5/DataSourceVMwareGuestInfo.py?format=TEXT | base64 -d > DataSourceVMwareGuestInfo.py
curl https://gerrit.mcp.mirantis.com/plugins/gitiles/kubernetes/vmware-guestinfo/+/refs/tags/v1.1.5/99-DataSourceVMwareGuestInfo.cfg?format=TEXT | base64 -d > 99-DataSourceVMwareGuestInfo.cfg

          2. Add 99-DataSourceVMwareGuestInfo.cfg to /etc/cloud/cloud.cfg.d/.

          3. Depending on the Python version on the VM operating system, add DataSourceVMwareGuestInfo.py to the cloud-init sources folder. Obtain the cloud-init folder on the OS:

            python -c 'import os; from cloudinit import sources; print(os.path.dirname(sources.__file__));'

        3. For Ubuntu, create /etc/cloud/cloud.cfg.d/99_mcc.cfg with the following content:

          datasource_list: [ VMware ]
package_update: false
package_upgrade: false
apt:
preserve_sources_list: true

      3. Optional. For RHEL, prepare the virt-who user configuration:

        Note

        For details about the virt-who user creation, see Prepare the VMware deployment user setup and permissions.

        1. Install virt-who:

          yum install virt-who -y
cp /etc/virt-who.d/template.conf /etc/virt-who.d/vcenter.conf

        2. Set up the file content using the following example:

          [vcenter]
type=esx
server=vcenter1.example.com
username=domain\vmware_read_only_username
encrypted_password=bd257f93d@482B76e6390cc54aec1a4d
owner=1234567
hypervisor_id=hostname
filter_hosts=esx1.example.com, esx2.example.com
          virt-who configuration parameters

          Parameter

          Description

          [vcenter]

          Name of the vCenter data center.

          type=esx

          Specifies the connection of the defined virt-who user to the vCenter Server.

          server

          The FQDN of the vCenter Server.

          username

          The virt-who user name on the vCenter Server with the read-only access.

          encrypted_password

          The virt-who password encrypted by the virt-who-password utility using the virt-who-password -p <password> command.

          owner

          The organization that the hypervisors belong to.

          hypervisor_id

          Specifies how to identify the hypervisors. Use a host name to provide meaningful host names to the Subscription Management. Alternatively, use uuid or hwuuid to avoid duplication in case of hypervisor renaming.

          filter_hosts

          List of hypervisors that never run RHEL VMs. Such hypervisors do not have to be reported by virt-who.

  7. For CentOS, verify that .yum mirrors are set to use only the *.centos.org URLs. Otherwise, access to other mirrors may be blocked by squid-proxy on managed clusters. For details, see Configure squid-proxy.

  8. For RHEL, remove the RHEL subscription from the node.

    subscription-manager remove --all
subscription-manager unregister
subscription-manager clean

  9. Configure the interface name for eth0:

    1. In /etc/default/grub, add the following parameters to GRUB_CMDLINE_LINUX:

      GRUB_CMDLINE_LINUX="net.ifnames=0 biosdevname=0"

    2. Update the GRUB configuration:

      update-grub

  10. Shut down the VM.

  11. Clone the VM to the template.

Now, proceed to Bootstrap a management cluster.