Expand IP addresses capacity in an existing cluster¶
If the subnet capacity on your existing cluster is not enough to add new
machines, use the l2TemplateSelector
feature to expand the IP addresses
capacity:
Create new
Subnet
object(s) to define additional address ranges for new machines.Set up routing between the existing and new subnets.
Create new L2 template(s) with the new subnet(s) being used in
l3Layout
.Set up
l2TemplateSelector
in theMachine
objects for new machines.
To expand IP addresses capacity for an existing cluster:
Verify the capacity of the subnet(s) currently associated with the L2 template(s) used for cluster deployment:
If
labelSelector
is not used for the given subnet, use thenamespace
value of the L2 template and thesubnetName
value from thel3Layout
section:kubectl get subnet -n <namespace> <subnetName>
If
labelSelector
is used for the given subnet, use thenamespace
value of the L2 template and comma-separated key-value pairs from thelabelSelector
section for the given subnet in thel3Layout
section:kubectl get subnet -n <namespace> -l <key1=value1>[<,key2=value2>...]
Example command:
kubectl get subnet -n test-ns -l cluster.sigs.k8s.io/cluster-name=managed123,user-defined/purpose=lcm-base
Example of system response:
NAME AGE CIDR GATEWAY CAPACITY ALLOCATABLE STATUS old-lcm-network 8d 192.168.1.0/24 192.168.1.1 253 0 OK
Existing Subnet example
apiVersion: "ipam.mirantis.com/v1alpha1" kind: Subnet metadata: name: old-lcm-network namespace: test-ns labels: cluster.sigs.k8s.io/cluster-name: managed123 ipam/SVC-k8s-lcm: "1" kaas.mirantis.com/provider: baremetal kaas.mirantis.com/region: region-one user-defined/purpose: lcm-base spec: cidr: 192.168.1.0/24 gateway: 192.168.1.1 . . . status: allocatable: 0 allocatedIPs: . . . capacity: 253 cidr: 192.168.1.0/24 gateway: 192.168.1.1 ranges: - 192.168.1.2-192.168.1.254
Note
The
kaas.mirantis.com/region
label is removed from all Container Cloud and MOSK objects in 24.1.Therefore, do not add the label starting with these releases. On existing clusters updated to these releases, or if added manually, Container Cloud ignores this label.
Existing L2 template example
apiVersion: ipam.mirantis.com/v1alpha1 kind: L2Template metadata: name: base-template namespace: test-ns labels: ipam/DefaultForCluster: "1" kaas.mirantis.com/provider: baremetal kaas.mirantis.com/region: region-one cluster.sigs.k8s.io/cluster-name: managed123 spec: autoIfMappingPrio: - provision - eno - ens l3Layout: - scope: namespace subnetName: lcm-subnet1 labelSelector: cluster.sigs.k8s.io/cluster-name: managed123 user-defined/purpose: lcm-base npTemplate: | version: 2 renderer: networkd ethernets: {{nic 0}}: match: macaddress: {{mac 0}} set-name: {{nic 0}} bridges: k8s-lcm: interfaces: - {{nic 0}} addresses: - {{ip "k8s-lcm:lcm-subnet1"}} gateway4: {{gateway_from_subnet "lcm-subnet1"}} nameservers: addresses: {{nameservers_from_subnet "lcm-subnet1"}}
Note
The
kaas.mirantis.com/region
label is removed from all Container Cloud and MOSK objects in 24.1.Therefore, do not add the label starting with these releases. On existing clusters updated to these releases, or if added manually, Container Cloud ignores this label.
Note
Before MOSK 23.3, an L2 template requires
clusterRef: <clusterName>
in thespec
section. Since MOSK 23.3, this parameter is deprecated and automatically migrated to thecluster.sigs.k8s.io/cluster-name: <clusterName>
label.Create new objects:
Subnet
with theuser-defined/purpose: lcm-additional
label.L2Template
with thealternative-template: “1”
label. The L2 template should reference the newSubnet
object using theuser-defined/purpose: lcm-additional
label in thelabelSelector
field.
Note
The label name
user-defined/purpose
is used for illustration purposes. Use any custom label name that differs from system names. Use of a unique prefix such asuser-defined/
is recommended.New subnet example
apiVersion: "ipam.mirantis.com/v1alpha1" kind: Subnet metadata: name: new-lcm-network namespace: test-ns labels: cluster.sigs.k8s.io/cluster-name: managed123 ipam/SVC-k8s-lcm: "1" kaas.mirantis.com/provider: baremetal kaas.mirantis.com/region: region-one user-defined/purpose: lcm-additional spec: cidr: 192.168.200.0/24 gateway: 192.168.200.1 . . . status: allocatable: 253 allocatedIPs: . . . capacity: 253 cidr: 192.168.200.0/24 gateway: 192.168.200.1 ranges: - 192.168.200.2-192.168.200.254
Note
The
kaas.mirantis.com/region
label is removed from all Container Cloud and MOSK objects in 24.1.Therefore, do not add the label starting with these releases. On existing clusters updated to these releases, or if added manually, Container Cloud ignores this label.
Alternative L2 template example
apiVersion: ipam.mirantis.com/v1alpha1 kind: L2Template metadata: name: alternative-template namespace: test-ns labels: kaas.mirantis.com/provider: baremetal kaas.mirantis.com/region: region-one cluster.sigs.k8s.io/cluster-name: managed123 alternative-template: “1” spec: autoIfMappingPrio: - provision - eno - ens l3Layout: - scope: namespace subnetName: lcm-subnet2 labelSelector: cluster.sigs.k8s.io/cluster-name: managed123 user-defined/purpose: lcm-additional npTemplate: | version: 2 renderer: networkd ethernets: {{nic 0}}: match: macaddress: {{mac 0}} set-name: {{nic 0}} bridges: k8s-lcm: interfaces: - {{nic 0}} addresses: - {{ip "k8s-lcm:lcm-subnet2"}} gateway4: {{gateway_from_subnet "lcm-subnet2"}} nameservers: addresses: {{nameservers_from_subnet "lcm-subnet2"}}
Note
The
kaas.mirantis.com/region
label is removed from all Container Cloud and MOSK objects in 24.1.Therefore, do not add the label starting with these releases. On existing clusters updated to these releases, or if added manually, Container Cloud ignores this label.
Note
Before MOSK 23.3, an L2 template requires
clusterRef: <clusterName>
in thespec
section. Since MOSK 23.3, this parameter is deprecated and automatically migrated to thecluster.sigs.k8s.io/cluster-name: <clusterName>
label.You can also reference the new
Subnet
object by using its name in thel3Layout
section of thealternative-template
L2 template.Snippet example of an alternative L2 template
... spec: ... l3Layout: - scope: namespace subnetName: new-lcm-network ... npTemplate: | ... bridges: k8s-lcm: interfaces: - {{nic 0}} addresses: - {{ip "k8s-lcm:new-lcm-network"}} gateway4: {{gateway_from_subnet "new-lcm-network"}} nameservers: addresses: {{nameservers_from_subnet "new-lcm-network"}}
Set up IP routing between the existing and new subnets using the tools of your cloud network infrastructure.
In the
providerSpec
section of the newMachine
object, define thealternative-template
label forl2TemplateSelector
:Snippet example of the new Machine object
apiVersion: cluster.k8s.io/v1alpha1 kind: Machine metadata: labels: cluster.sigs.k8s.io/cluster-name: managed123 kaas.mirantis.com/provider: baremetal kaas.mirantis.com/region: region-one name: additional-machine namespace: test-ns spec: ... providerSpec: value: ... l2TemplateSelector: label: alternative-template
Note
The
kaas.mirantis.com/region
label is removed from all Container Cloud and MOSK objects in 24.1.Therefore, do not add the label starting with these releases. On existing clusters updated to these releases, or if added manually, Container Cloud ignores this label.
After creation, the new machine will use the alternative L2 template that uses the
new-lcm-network
subnet linked byL3Layout
.Deprecated since MOSK 24.2. Define additional address ranges for MetalLB. For details, see the optional step for the MetalLB service in Create subnets.
You can create one or several
Subnet
objects to extend the MetalLB address pool with additional ranges. When the MetalLB traffic is routed through the default gateway, you can add the MetalLB address ranges that belong to different CIDR subnet addresses.For example:
Snippet example of the MetalLB configuration
apiVersion: ipam.mirantis.com/v1alpha1 kind: Subnet metadata: labels: cluster.sigs.k8s.io/cluster-name: managed123 ipam/SVC-MetalLB: '1' kaas.mirantis.com/region: region-one kaas.mirantis.com/provider: baremetal user-defined/purpose: metallb-additional1 name: metallb-svc1-for-managed namespace: test-ns spec: cidr: 172.16.168.0/24 includeRanges: - 172.16.168.11-172.16.168.30 - 172.16.168.41-172.16.168.50 apiVersion: ipam.mirantis.com/v1alpha1 kind: Subnet metadata: labels: cluster.sigs.k8s.io/cluster-name: managed123 ipam/SVC-MetalLB: '1' kaas.mirantis.com/region: region-one kaas.mirantis.com/provider: baremetal user-defined/purpose: metallb-additional2 name: metallb-svc2-for-managed namespace: test-ns spec: cidr: 172.16.169.0/24 includeRanges: - 172.16.169.11-172.16.169.20
Note
The
kaas.mirantis.com/region
label is removed from all Container Cloud and MOSK objects in 24.1.Therefore, do not add the label starting with these releases. On existing clusters updated to these releases, or if added manually, Container Cloud ignores this label.
Verify the created objects for MetalLB.
For reference, use
managed-ns_Subnet_metallb-public-for-managed.yaml
in Container Cloud documentation: Example of a complete L2 templates configuration for cluster creation.