Create an L2 template for a new cluster

After you create subnets for one or more MOSK clusters or projects as described in Create subnets, follow the procedure below to create L2 templates for a MOSK cluster.

L2 templates are used directly during provisioning. This way, a hardware node obtains and applies a complete network configuration during the first system boot.

Caution

Update any L2 template created before Container Cloud 2.9.0 (Cluster releases 6.14.0, 5.15.0, or earlier) to the new format as described in Container Cloud Release Notes: Switch L2 templates to the new format.

Сreate an L2 template for a new MOSK cluster

Caution

Create L2 templates before adding any machines to your new MOSK cluster.

  1. Log in to a local machine where your management cluster kubeconfig is located and where kubectl is installed.

    Note

    The management cluster kubeconfig is created during the last stage of the management cluster bootstrap.

  2. Create a set of L2Template YAML files specific to your deployment using exemplary templates provided in Create L2 templates.

    Note

    You can create several L2 templates with different configurations to be applied to different nodes of the same cluster. See Assign L2 templates to machines for details.

  3. Add or edit the mandatory labels and parameters in the new L2 template. For description of mandatory labels and parameters, see Container Cloud API Reference: L2Template.

  4. Optional. To designate an L2 template as default, assign the ipam/DefaultForCluster label to it. Only one L2 template in a cluster can have this label. It will be used for machines that do not have an L2 template explicitly assigned to them.

    Note

    You may skip this step and add the default label along with other custom labels using the Container Cloud web UI, as described below in this procedure.

    To assign the default template to the cluster:

    Use the mandatory cluster.sigs.k8s.io/cluster-name label in the L2 template metadata section.

    Use the cluster.sigs.k8s.io/cluster-name label or the clusterRef parameter in the L2 template spec section. During cluster update to 2.25.0, this deprecated parameter is automatically migrated to the cluster.sigs.k8s.io/cluster-name label.

  5. Optional. Add custom labels to the L2 template. You can refer to these labels to assign the L2 template to machines.

  6. Add the L2 template to your management cluster. Select one of the following options:

    kubectl --kubeconfig <pathToManagementClusterKubeconfig> apply -f <pathToL2TemplateYamlFile>

    1. Log in to the Container Cloud web UI with the m:kaas:namespace@operator or m:kaas:namespace@writer permissions.

    2. Switch to the required non-default project using the Switch Project action icon located on top of the main left-side navigation panel.

      Caution

      Do not create a MOSK cluster in the default project (Kubernetes namespace), which is dedicated for the management cluster only. If no projects are defined, first create a new mosk project as described in Create a project for MOSK clusters.

    3. In the left sidebar, navigate to Networks and click the L2 Templates tab.

    4. Click Create L2 Template.

    5. Fill out the Create L2 Template form as required:

      • Name

        L2 template name.

      • Cluster

        Cluster name that the L2 template is being added for. To set the L2 template as default for all machines, also select Set default for the cluster.

      • Specification

        L2 specification in the YAML format that you have previously created. Click Edit to edit the L2 template if required.

        Note

        Before Container Cloud 2.28.0 (Cluster releases 17.3.0 and 16.3.0), the field name is YAML file, and you can upload the required YAML file instead of inserting and editing it.

      • Labels

        Available since Container Cloud 2.28.0 (Cluster releases 17.3.0 and 16.3.0). Key-value pairs attached to the L2 template. For details, see Container Cloud API Reference: L2Template metadata.

  7. Optional. Further modify the template, if required. For description of parameters, see Container Cloud API Reference: L2Template resource.

    kubectl --kubeconfig <pathToManagementClusterKubeconfig> \
-n <ProjectNameForNewMOSKCluster> edit l2template <L2templateName>

    Caution

    Modification of L2 templates in use is only allowed with a mandatory validation step from the infrastructure operator to prevent accidental cluster failures due to unsafe changes. The list of risks posed by modifying L2 templates includes:

    • Services running on hosts cannot reconfigure automatically to switch to the new IP addresses and/or interfaces.

    • Connections between services are interrupted unexpectedly, which can cause data loss.

    • Incorrect configurations on hosts can lead to irrevocable loss of connectivity between services and unexpected cluster partition or disassembly.

    For details, see Modify network configuration on an existing machine.

  8. Proceed with Add a machine. The resulting L2 template will be used to render the netplan configuration for the MOSK cluster machines.

Workflow of the netplan configuration using an L2 template

  1. The kaas-ipam service uses the data from BareMetalHost, L2Template, and Subnet objects to generate the netplan configuration for every cluster machine.

    Note

    Before update of the management cluster to Container Cloud 2.29.0 (Cluster release 16.4.0), instead of BareMetalHostInventory, use the BareMetalHost object. For details, see Container Cloud API Reference: BareMetalHost resource.

    Caution

    While the Cluster release of the management cluster is 16.4.0, BareMetalHostInventory operations are allowed to m:kaas@management-admin only. This limitation is lifted once the management cluster is updated to the Cluster release 16.4.1 or later.

  2. The generated netplan configuration is saved in the status.netconfigFiles section of the IpamHost object. If the status.netconfigFilesState field of the IpamHost object is OK, the configuration was rendered in the IpamHost object successfully. Otherwise, the status contains an error message.

    Caution

    The following fields of the ipamHost status are renamed since MOSK 23.1 in the scope of the L2Template and IpamHost objects refactoring:

    • netconfigV2 to netconfigCandidate

    • netconfigV2state to netconfigCandidateState

    • netconfigFilesState to netconfigFilesStates (per file)

    No user actions are required after renaming.

    The format of netconfigFilesState changed after renaming. The netconfigFilesStates field contains a dictionary of statuses of network configuration files stored in netconfigFiles. The dictionary contains the keys that are file paths and values that have the same meaning for each file that netconfigFilesState had:

    • For a successfully rendered configuration file: OK: <timestamp> <sha256-hash-of-rendered-file>, where a timestamp is in the RFC 3339 format.

    • For a failed rendering: ERR: <error-message>.

  3. The baremetal-provider service copies data from status.netconfigFiles of the IpamHost object to the Spec.StateItemsOverwrites[‘deploy’][‘bm_ipam_netconfigv2’] parameter of LCMMachine.

  4. The lcm-agent service on every host synchronizes the LCMMachine data to its host. The lcm-agent service runs a playbook to update the netplan configuration on the host during the pre-download and deploy phases.