L2Template¶
This section describes the L2Template
resource used in Mirantis
Container Cloud API.
By default, Container Cloud configures a single interface on cluster nodes,
leaving all other physical interfaces intact.
With L2Template
, you can create advanced host networking configurations
for your clusters. For example, you can create bond interfaces on top of
physical interfaces on the host.
For demonstration purposes, the Container Cloud L2Template
custom resource (CR) is split into the following major sections:
L2Template metadata¶
The Container Cloud L2Template
CR contains the following fields:
apiVersion
API version of the object that is
ipam.mirantis.com/v1alpha1
.
kind
Object type that is
L2Template
.
metadata
The
metadata
field contains the following subfields:name
Name of the
L2Template
object.
namespace
Project in which the
L2Template
object was created.
labels
Key-value pairs that are attached to the object:
Caution
All
ipam/*
labels, exceptipam/DefaultForCluster
, are set automatically and must not be configured manually.cluster.sigs.k8s.io/cluster-name
Mandatory for newly created
L2Template
since Container Cloud 2.25.0 (Cluster releases 17.0.0 and 16.0.0). References theCluster
object name that this template is applied to.The process of selecting the
L2Template
object for a specific cluster is as follows:The
kaas-ipam
controller monitors theL2Template
objects with thecluster.sigs.k8s.io/cluster-name: <clusterName>
label.The
L2Template
object with thecluster.sigs.k8s.io/cluster-name: <clusterName>
label is assigned to a cluster withName: <clusterName>
, if available.
ipam/PreInstalledL2Template: "1"
Is automatically added during a management cluster deployment. Indicates that the current
L2Template
object was preinstalled. Represents L2 templates that are automatically copied to a project once it is created. Once the L2 templates are copied, theipam/PreInstalledL2Template
label is removed.Note
Preinstalled L2 templates are removed in Container Cloud 2.26.0 (Cluster releases 17.1.0 and 16.1.0) along with the
ipam/PreInstalledL2Template
label. During cluster update to the mentioned releases, existing preinstalled templates are automatically removed.
ipam/DefaultForCluster
This label is unique per cluster. When you use several L2 templates per cluster, only the first template is automatically labeled as the default one. All consequent templates must be referenced in the machines configuration files using
L2templateSelector
. You can manually configure this label if required.
ipam/UID
Unique ID of an object.
kaas.mirantis.com/provider
Provider type.
kaas.mirantis.com/region
Region name.
Note
The
kaas.mirantis.com/region
label is removed from all Container Cloud objects in 2.26.0 (Cluster releases 17.1.0 and 16.1.0). Therefore, do not add the label starting these releases. On existing clusters updated to these releases, or if manually added, this label will be ignored by Container Cloud.
Warning
Labels and annotations that are not documented in this API Reference are generated automatically by Container Cloud. Do not modify them using the Container Cloud API.
Configuration example:
apiVersion: ipam.mirantis.com/v1alpha1
kind: L2Template
metadata:
name: l2template-test
namespace: default
labels:
ipam/DefaultForCluster: "1"
cluster.sigs.k8s.io/cluster-name: test-cluster
kaas.mirantis.com/provider: baremetal
L2Template configuration¶
L2 template requirements
An L2 template must have the same project (Kubernetes namespace) as the referenced cluster.
A cluster can be associated with many L2 templates. Only one of them can have the
ipam/DefaultForCluster
label. Every L2 template that does not have theipam/DefaultForCluster
label can be later assigned to a particular machine usingl2TemplateSelector
.The following rules apply to the default L2 template of a namespace:
Since Container Cloud 2.25.0 (Cluster releases 17.0.0 and 16.0.0), creation of the default L2 template for a namespace is disabled. On existing clusters, the
Spec.clusterRef: default
parameter of such an L2 template is automatically removed during the migration process. Subsequently, this parameter is not substituted with thecluster.sigs.k8s.io/cluster-name
label, ensuring the application of the L2 template across the entire Kubernetes namespace. Therefore, you can continue using existing default namespaced L2 templates.Before Container Cloud 2.25.0 (Cluster releases 15.x, 14.x, or earlier), the default
L2Template
object of a namespace must have theSpec.clusterRef: default
parameter that is deprecated since 2.25.0.
The spec
field of the L2Template
resource describes the desired
state of the object. It contains the following fields:
ifMapping
List of interface names for the template. The interface mapping is defined globally for all
Machine
objects linked to the template but can be overridden at the host level, if required, by editing theIpamHost
object for a particular host. TheifMapping
parameter is mutually exclusive withautoIfMappingPrio
.
autoIfMappingPrio
List of prefixes, such as
eno
,ens
, and so on, to match the interfaces to automatically create a list for the template. If you are not aware of any specific ordering of interfaces on the nodes, use the default ordering from Predictable Network Interfaces Names specification for systemd.You can also override the default NIC list per host using the
IfMappingOverride
parameter of the correspondingIpamHost
. Theprovision
value corresponds to the network interface that was used to provision a node. Usually, it is the first NIC found on a particular node. It is defined explicitly to ensure that this interface will not be reconfigured accidentally.The
autoIfMappingPrio
parameter is mutually exclusive withifMapping
.
l3Layout
Subnets to be used in the
npTemplate
section. The field contains a list of subnet definitions with parameters used by template macros.subnetName
Defines the alias name of the subnet that can be used to reference this subnet from the template macros. This parameter is mandatory for every entry in the
l3Layout
list.
subnetPool
Unsupported since 2.28.0 (17.3.0 and 16.3.0)Optional. Default: none. Defines a name of the parent
SubnetPool
object that will be used to create aSubnet
object with a givensubnetName
andscope
. For deprecation details, see MOSK Deprecation Notes: SubnetPool resource management.If a corresponding
Subnet
object already exists, nothing will be created and the existing object will be used. If noSubnetPool
is provided, no newSubnet
object will be created.
scope
Logical scope of the
Subnet
object with a correspondingsubnetName
. Possible values:global
- theSubnet
object is accessible globally, for any Container Cloud project and cluster, for example, the PXE subnet.namespace
- theSubnet
object is accessible within the same project where the L2 template is defined.cluster
- Unsupported since Container Cloud 2.28.0 (Cluster releases 17.3.0 and 16.3.0). TheSubnet
object uses the namespace where the referenced cluster is located. A subnet is only accessible to the cluster thatL2Template.metadata.labels:cluster.sigs.k8s.io/cluster-name
(mandatory since MOSK 23.3) orL2Template.spec.clusterRef
(deprecated in MOSK 23.3) refers to. TheSubnet
objects with thecluster
scope will be created for every new cluster.
Note
Every subnet referenced in an L2 template can have either a global or namespaced scope. In the latter case, the subnet must exist in the same project where the corresponding cluster and L2 template are located.
labelSelector
Contains a dictionary of labels and their respective values that will be used to find the matching
Subnet
object. If thelabelSelector
field is omitted, theSubnet
object will be selected by name, specified by thesubnetName
parameter.Caution
The labels and their values in this section must match the ones added for the corresponding
Subnet
object.
Caution
The
l3Layout
section is mandatory for eachL2Template
custom resource.
npTemplate
A netplan-compatible configuration with special lookup functions that defines the networking settings for the cluster hosts, where physical NIC names and details are parameterized. This configuration will be processed using Go templates. Instead of specifying IP and MAC addresses, interface names, and other network details specific to a particular host, the template supports use of special lookup functions. These lookup functions, such as
nic
,mac
,ip
, and so on, return host-specific network information when the template is rendered for a particular host.Caution
All rules and restrictions of the netplan configuration also apply to L2 templates. For details, see official netplan documentation.
Caution
Mirantis strongly recommends following the below conventions on network interface naming:
A physical NIC name set by an L2 template must not exceed 15 symbols. Otherwise, an L2 template creation fails. This limit is set by the Linux kernel.
Names of virtual network interfaces such as VLANs, bridges, bonds, veth, and so on must not exceed 15 symbols.
Mirantis recommends setting interfaces names that do not exceed 13 symbols for both physical and virtual interfaces to avoid corner cases and issues in netplan rendering.
The following table describes the main lookup functions for an L2 template.
Lookup function
Description
{{nic N}}
Name of a NIC number N. NIC numbers correspond to the interface mapping list. This macro can be used as a key for the elements of the
ethernets
map, or as the value of thename
andset-name
parameters of a NIC. It is also used to reference the physical NIC from definitions of virtual interfaces (vlan
,bridge
).{{mac N}}
MAC address of a NIC number N registered during a host hardware inspection.
{{ip “N:subnet-a”}}
IP address and mask for a NIC number N. The address will be auto-allocated from the given subnet if the address does not exist yet.
{{ip “br0:subnet-x”}}
IP address and mask for a virtual interface,
“br0”
in this example. The address will be auto-allocated from the given subnet if the address does not exist yet.For virtual interfaces names, an IP address placeholder must contain a human-readable ID that is unique within the L2 template and must have the following format:
{{ip "<shortUniqueHumanReadableID>:<subnetNameFromL3Layout>"}}
The
<shortUniqueHumanReadableID>
is made equal to a virtual interface name throughout this document and Container Cloud bootstrap templates.{{cidr_from_subnet “subnet-a”}}
IPv4 CIDR address from the given subnet.
{{gateway_from_subnet “subnet-a”}}
IPv4 default gateway address from the given subnet.
{{nameservers_from_subnet “subnet-a”}}
List of the IP addresses of name servers from the given subnet.
{{cluster_api_lb_ip}}
Technology Preview since Container Cloud 2.24.4 (Cluster releases 15.0.3 and 14.0.3). IP address for a cluster API load balancer.
clusterRef
Caution
Deprecated since Container Cloud 2.25.0 (Cluster releases 17.0.0 and 16.0.0) in favor of the mandatory
cluster.sigs.k8s.io/cluster-name
label. Will be removed in one of the following releases.On existing clusters, this parameter is automatically migrated to the
cluster.sigs.k8s.io/cluster-name
label since 2.25.0.If an existing cluster has
clusterRef: default
set, the migration process involves removing this parameter. Subsequently, it is not substituted with thecluster.sigs.k8s.io/cluster-name
label, ensuring the application of the L2 template across the entire Kubernetes namespace.The
Cluster
object name that this template is applied to. Thedefault
value is used to apply the given template to all clusters within a particular project, unless an L2 template that references a specific cluster name exists. TheclusterRef
field has priority over thecluster.sigs.k8s.io/cluster-name
label:When
clusterRef
is set to a non-default
value, thecluster.sigs.k8s.io/cluster-name
label will be added or updated with that value.When
clusterRef
is set todefault
, thecluster.sigs.k8s.io/cluster-name
label will be absent or removed.
Configuration example:
spec:
autoIfMappingPrio:
- provision
- eno
- ens
- enp
l3Layout:
- subnetName: kaas-mgmt
scope: global
labelSelector:
kaas-mgmt-subnet: ""
- subnetName: demo-pods
scope: namespace
- subnetName: demo-ext
scope: namespace
- subnetName: demo-ceph-cluster
scope: namespace
- subnetName: demo-ceph-replication
scope: namespace
npTemplate: |
version: 2
ethernets:
{{nic 1}}:
dhcp4: false
dhcp6: false
addresses:
- {{ip "1:kaas-mgmt"}}
gateway4: {{gateway_from_subnet "kaas-mgmt"}}
nameservers:
addresses: {{nameservers_from_subnet "kaas-mgmt"}}
match:
macaddress: {{mac 1}}
set-name: {{nic 1}}
L2Template status¶
The status
field of the L2Template
resource reflects the actual state
of the L2Template
object and contains the following fields:
state
Since 2.23.0Message that reflects the current status of the resource. The list of possible values includes the following:
OK
- object is operational.ERR
- object is non-operational. This status has a detailed description in themessages
list.TERM
- object was deleted and is terminating.
messages
Since 2.23.0List of error or warning messages if the object state is
ERR
.
objCreated
Date, time, and IPAM version of the resource creation.
objStatusUpdated
Date, time, and IPAM version of the last update of the
status
field in the resource.
objUpdated
Date, time, and IPAM version of the last resource update.
phase
Deprecated since Container Cloud 2.23.0 (Cluster release 11.7.0) and will be removed in one of the following releases in favor of
state
. Possible values:Active
,Failed
, orTerminating
.
reason
Deprecated since Container Cloud 2.23.0 (Cluster release 11.7.0) and will be removed in one of the following releases in favor of
messages
. For the field description, seemessages
.
Configuration example:
status:
phase: Failed
state: ERR
messages:
- "ERR: The kaas-mgmt subnet in the terminating state."
objCreated: 2021-10-21T19:09:32Z by v5.1.0-20210930-121522-f5b2af8
objStatusUpdated: 2021-10-21T19:14:18.748114886Z by v5.1.0-20210930-121522-f5b2af8
objUpdated: 2021-10-21T19:09:32.606968024Z by v5.1.0-20210930-121522-f5b2af8