Bootstrap a management cluster¶
After you complete the prerequisite steps described in Prerequisites, proceed with bootstrapping your VMware vSphere-based Mirantis Container Cloud management cluster.
To bootstrap a vSphere-based management cluster:
Log in to the bootstrap node running Ubuntu 20.04 that is configured as described in Prerequisites.
Prepare the bootstrap script:
Download and run the Container Cloud bootstrap script:
apt install wget wget https://binary.mirantis.com/releases/get_container_cloud.sh chmod 0755 get_container_cloud.sh ./get_container_cloud.sh
Change the directory to the
kaas-bootstrap
folder created by the script.
Obtain your license file that will be required during the bootstrap:
Create a user account at www.mirantis.com.
Log in to your account and download the
mirantis.lic
license file.Save the license file as
mirantis.lic
under thekaas-bootstrap
directory on the bootstrap node.Verify that
mirantis.lic
contains the exact Container Cloud license previously downloaded from www.mirantis.com by decoding the license JWT token, for example, using jwt.io.Example of a valid decoded Container Cloud license data with the mandatory
license
field:{ "exp": 1652304773, "iat": 1636669973, "sub": "demo", "license": { "dev": false, "limits": { "clusters": 10, "workers_per_cluster": 10 }, "openstack": null } }
Warning
The MKE license does not apply to
mirantis.lic
. For details about MKE license, see MKE documentation.
Prepare deployment templates:
Modify
templates/vsphere/vsphere-config.yaml.template
:Note
Contact your vSphere administrator to provide you with the below parameters.
vSphere configuration data¶ Parameter
Description
SET_VSPHERE_SERVER
IP address or FQDN of the vCenter Server.
SET_VSPHERE_SERVER_PORT
Port of the vCenter Server. For example,
port: "8443"
. Leave empty to use"443"
by default.SET_VSPHERE_DATACENTER
vSphere data center name.
SET_VSPHERE_SERVER_INSECURE
Flag that controls validation of the vSphere Server certificate. Must be
true
orfalse
.SET_VSPHERE_CAPI_PROVIDER_USERNAME
vSphere Cluster API provider user name that you added when preparing the deployment user setup and permissions.
SET_VSPHERE_CAPI_PROVIDER_PASSWORD
vSphere Cluster API provider user password.
SET_VSPHERE_CLOUD_PROVIDER_USERNAME
vSphere Cloud Provider deployment user name that you added when preparing the deployment user setup and permissions.
SET_VSPHERE_CLOUD_PROVIDER_PASSWORD
vSphere Cloud Provider deployment user password.
Modify the
templates/vsphere/cluster.yaml.template
parameters:Modify the following required network parameters:
Required parameters¶ Parameter
Description
SET_LB_HOST
IP address from the provided vSphere network for load balancer (Keepalived).
SET_VSPHERE_METALLB_RANGE
MetalLB range of IP addresses that can be assigned to load balancers for Kubernetes Services.
SET_VSPHERE_DATASTORE
Name of the vSphere datastore. You can use different datastores for vSphere Cluster API and vSphere Cloud Provider.
SET_VSPHERE_MACHINES_FOLDER
Path to a folder where the cluster machines metadata will be stored.
SET_VSPHERE_NETWORK_PATH
Path to a network for cluster machines.
SET_VSPHERE_RESOURCE_POOL_PATH
Path to a resource pool in which VMs will be created.
Note
To obtain the
LB_HOST
andVSPHERE_METALLB_RANGE
parameters for the selected vSphere network, contact your vSphere administrator who provides you with IP ranges dedicated to your environment only.Modify other parameters if required. For example, add the corresponding values for
cidrBlocks
in thespec::clusterNetwork::services
section.For either DHCP or non-DHCP vSphere network:
Determine the vSphere network parameters as described in VMware vSphere network objects and IPAM recommendations.
Provide the following additional parameters for a proper network setup on machines using embedded IP address management (IPAM) in
templates/vsphere/cluster.yaml.template
Note
To obtain IPAM parameters for the selected vSphere network, contact your vSphere administrator who provides you with IP ranges dedicated to your environment only.
vSphere configuration data¶ Parameter
Description
ipamEnabled
Enables IPAM. Recommended value is
true
for either DHCP or non-DHCP networks.SET_VSPHERE_NETWORK_CIDR
CIDR of the provided vSphere network. For example,
10.20.0.0/16
.SET_VSPHERE_NETWORK_GATEWAY
Gateway of the provided vSphere network.
SET_VSPHERE_CIDR_INCLUDE_RANGES
IP range for the cluster machines. Specify the range of the provided CIDR. For example,
10.20.0.100-10.20.0.200
. If the DHCP network is used, this range must not intersect with the DHCP range of the network.SET_VSPHERE_CIDR_EXCLUDE_RANGES
Optional. IP ranges to be excluded from being assigned to the cluster machines. The MetalLB range and
SET_LB_HOST
should not intersect with the addresses for IPAM. For example,10.20.0.150-10.20.0.170
.SET_VSPHERE_NETWORK_NAMESERVERS
List of nameservers for the provided vSphere network.
For RHEL deployments, fill out
templates/vsphere/rhellicenses.yaml.template
using one of the following set of parameters for RHEL machines subscription:The user name and password of your RedHat Customer Portal account associated with your RHEL license for Virtual Datacenters.
Optionally, provide the subscription allocation pools to use for the RHEL subscription activation. If not needed, remove the
poolIDs
field forsubscription-manager
to automatically select the licenses for machines.For example:
spec: username: <username> password: value: <password> poolIDs: - <pool1> - <pool2>
The activation key and organization ID associated with your RedHat account with RHEL license for Virtual Datacenters. The activation key can be created by the organization administrator on the RedHat Customer Portal.
If you use the RedHat Satellite server for management of your RHEL infrastructure, you can provide a pre-generated activation key from that server. In this case:
Provide the URL to the RedHat Satellite RPM for installation of the CA certificate that belongs to that server.
Configure
squid-proxy
on the management or regional cluster to allow access to your Satellite server. For details, see Configure squid-proxy.
For example:
spec: activationKey: value: <activation key> orgID: "<organization ID>" rpmUrl: <rpm url>
Caution
For RHEL 8.4 TechPreview, verify mirrors configuration for your activation key. For more details, see RHEL 8 mirrors configuration.
Warning
Provide only one set of parameters. Mixing the parameters from different activation methods will cause deployment failure.
For CentOS deployments, in
templates/vsphere/rhellicenses.yaml.template
, remove all lines underitems:
.
In
bootstrap.env
, add theKAAS_VSPHERE_ENABLED=true
environment variable that enables the vSphere provider deployment in Container Cloud.Configure NTP server.
Before Container Cloud 2.23.0, optional if servers from the Ubuntu NTP pool (
*.ubuntu.pool.ntp.org
) are accessible from the node where the management cluster is being provisioned. Otherwise, configure the regional NTP server parameters as described below.Since Container Cloud 2.23.0, optionally disable NTP that is enabled by default. This option disables the management of
chrony
configuration by Container Cloud to use your own system forchrony
management. Otherwise, configure the regional NTP server parameters as described below.NTP configuration
Configure the regional NTP server parameters to be applied to all machines of regional and managed clusters in the specified region.
In
templates/vsphere/cluster.yaml.template
, add thentp:servers
section with the list of required server names:spec: ... providerSpec: value: kaas: ... ntpEnabled: true regional: - helmReleases: - name: <providerName>-provider values: config: lcm: ... ntp: servers: - 0.pool.ntp.org ... provider: <providerName> ...
To disable NTP:
spec: ... providerSpec: value: ... ntpEnabled: false ...
Prepare the VM template as described in Prepare the virtual machine template.
In
templates/vsphere/machines.yaml.template
, define the following parameters:rhelLicense
RHEL license name defined in
rhellicenses.yaml.template
, defaults tokaas-mgmt-rhel-license
. Remove or comment out this parameter for CentOS and Ubuntu deployments.
diskGiB
Disk size in GiB for machines that must match the disk size of the VM template. You can leave this parameter commented to use the disk size of the VM template. The minimum requirement is 120 GiB.
template
Path to the VM template prepared in the previous step.
Sample template:
spec: providerSpec: value: apiVersion: vsphere.cluster.k8s.io/v1alpha1 kind: VsphereMachineProviderSpec rhelLicense: <rhelLicenseName> numCPUs: 8 memoryMiB: 24576 # diskGiB: 120 template: <vSphereVMTemplatePath>
Also, modify other parameters if required.
If you require all Internet access to go through a proxy server, in
bootstrap.env
, add the following environment variables to bootstrap the management and regional cluster using proxy:HTTP_PROXY
HTTPS_PROXY
NO_PROXY
PROXY_CA_CERTIFICATE_PATH
Example snippet:
export HTTP_PROXY=http://proxy.example.com:3128 export HTTPS_PROXY=http://user:pass@proxy.example.com:3128 export NO_PROXY=172.18.10.0,registry.internal.lan export PROXY_CA_CERTIFICATE_PATH="/home/ubuntu/.mitmproxy/mitmproxy-ca-cert.cer"
The following formats of variables are accepted:
Proxy configuration data¶ Variable
Format
HTTP_PROXY
HTTPS_PROXY
http://proxy.example.com:port
- for anonymous access.http://user:password@proxy.example.com:port
- for restricted access.
NO_PROXY
Comma-separated list of IP addresses or domain names. Mandatory to add
host[:port]
of the vCenter server.PROXY_CA_CERTIFICATE_PATH
Optional. Absolute path to the proxy CA certificate for man-in-the-middle (MITM) proxies. Must be placed on the bootstrap node to be trusted. For details, see Install a CA certificate for a MITM proxy on a bootstrap node.
Warning
If you require Internet access to go through a MITM proxy, ensure that the proxy has streaming enabled as described in Enable streaming for MITM.
Note
For MOSK-based deployments, the parameter is generally available since MOSK 22.4.
For implementation details, see Proxy and cache support.
Caution
In MITM proxy deployments, use the internal Red Hat Satellite server to register RHEL machines so that a VM can access this server directly without a MITM proxy.
For the list of Mirantis resources and IP addresses to be accessible from the Container Cloud clusters, see Requirements for a VMware vSphere-based cluster.
Optional. Configure external identity provider for IAM.
Optional. Enable infinite timeout for all bootstrap stages by exporting the following environment variable or adding it to
bootstrap.env
:export KAAS_BOOTSTRAP_INFINITE_TIMEOUT=true
Infinite timeout prevents the bootstrap failure due to timeout. This option is useful in the following cases:
The network speed is slow for artifacts downloading
An infrastructure configuration does not allow booting fast
A bare-metal node inspecting presupposes more than two HDDSATA disks to attach to a machine
Optional. Available since Container Cloud 2.23.0. Customize the cluster and region name by exporting the following environment variables or adding them to
bootstrap.env
:export REGION=<customRegionName> export CLUSTER_NAME=<customClusterName>
By default, the system uses
region-one
for the region name andkaas-mgmt
for the management cluster name.Run the bootstrap script:
./bootstrap.sh all
In case of deployment issues, refer to Troubleshooting and inspect logs.
If the script fails for an unknown reason:
Run the cleanup script:
./bootstrap.sh cleanup
Rerun the bootstrap script.
When the bootstrap is complete, collect and save the following management cluster details in a secure location:
The
kubeconfig
file located in the same directory as the bootstrap script. This file contains the admin credentials for the management cluster.The private
ssh_key
for access to the management cluster nodes that is located in the same directory as the bootstrap script.Note
If the initial version of your Container Cloud management cluster was earlier than 2.6.0,
ssh_key
is namedopenstack_tmp
and is located at~/.ssh/
.The URL for the Container Cloud web UI.
To create users with permissions required for accessing the Container Cloud web UI, see Create initial users after a management cluster bootstrap.
The StackLight endpoints. For details, see Access StackLight web UIs.
The Keycloak URL that the system outputs when the bootstrap completes. The admin password for Keycloak is located in
kaas-bootstrap/passwords.yml
along with other IAM passwords.
Note
The Container Cloud web UI and StackLight endpoints are available through Transport Layer Security (TLS) and communicate with Keycloak to authenticate users. Keycloak is exposed using HTTPS and self-signed TLS certificates that are not trusted by web browsers.
To use your own TLS certificates for Keycloak, refer to Configure TLS certificates for cluster applications.
Note
When the bootstrap is complete, the bootstrap cluster resources are freed up.
Now, you can proceed with operating your management cluster using the Container Cloud web UI and deploying managed clusters as described in Create and operate a VMware vSphere-based managed cluster.