Create subnets¶
Before creating an L2 template, ensure that you have the required subnets
that can be used in the L2 template to allocate IP addresses for the
managed cluster nodes.
Where required, create a number of subnets for a particular project
using the Subnet
CR. A subnet has three logical scopes:
global - CR uses the
default
namespace. A subnet can be used for any cluster located in any project.namespaced - CR uses the namespace that corresponds to a particular project where managed clusters are located. A subnet can be used for any cluster located in the same project.
cluster - CR uses the namespace where the referenced cluster is located. A subnet is only accessible to the cluster that
L2Template.spec.clusterRef
refers to. TheSubnet
objects with thecluster
scope will be created for every new cluster.
You can have subnets with the same name in different projects. In this case, the subnet that has the same project as the cluster will be used. One L2 template may often reference several subnets, those subnets may have different scopes in this case.
The IP address objects (IPaddr
CR) that are allocated from subnets
always have the same project as their corresponding IpamHost
objects,
regardless of the subnet scope.
To create subnets for a cluster:
Log in to a local machine where your management cluster
kubeconfig
is located and wherekubectl
is installed.Note
The management cluster
kubeconfig
is created during the last stage of the management cluster bootstrap.Create the
subnet.yaml
file with a number of global or namespaced subnets depending on the configuration of your cluster:kubectl --kubeconfig <pathToManagementClusterKubeconfig> apply -f <SubnetFileName.yaml>
Note
In the command above and in the steps below, substitute the parameters enclosed in angle brackets with the corresponding values.
Example of a
subnet.yaml
file:apiVersion: ipam.mirantis.com/v1alpha1 kind: Subnet metadata: name: demo namespace: demo-namespace spec: cidr: 10.11.0.0/24 gateway: 10.11.0.9 includeRanges: - 10.11.0.5-10.11.0.70 nameservers: - 172.18.176.6
Specification fields of the Subnet object¶ Parameter
Description
cidr
(singular)A valid IPv4 CIDR, for example, 10.11.0.0/24.
includeRanges
(list)A list of IP address ranges within the given CIDR that should be used in the allocation of IPs for nodes (excluding the gateway address). The IPs outside the given ranges will not be used in the allocation. Each element of the list can be either an interval 10.11.0.5-10.11.0.70 or a single address 10.11.0.77. In the example above, the addresses
10.11.0.5-10.11.0.70
(excluding the gateway address 10.11.0.9) will be allocated for nodes. TheincludeRanges
parameter is mutually exclusive withexcludeRanges
.excludeRanges
(list)A list of IP address ranges within the given CIDR that should not be used in the allocation of IPs for nodes. The IPs within the given CIDR but outside the given ranges will be used in the allocation (excluding gateway address). Each element of the list can be either an interval 10.11.0.5-10.11.0.70 or a single address 10.11.0.77. The
excludeRanges
parameter is mutually exclusive withincludeRanges
.useWholeCidr
(boolean)If set to
true
, the subnet address (10.11.0.0 in the example above) and the broadcast address (10.11.0.255 in the example above) are included into the address allocation for nodes. Otherwise, (false
by default), the subnet address and broadcast address will be excluded from the address allocation.gateway
(singular)A valid gateway address, for example, 10.11.0.9.
nameservers
(list)A list of the IP addresses of name servers. Each element of the list is a single address, for example, 172.18.176.6.
Caution
The subnet for the PXE network is automatically created during deployment and must contain the
ipam/DefaultSubnet: "1"
label. Each bare metal region must have only one subnet with this label.Caution
You may use different subnets to allocate IP addresses to diffrenet components of MCC in your cluster. See below for detailed list of available options. Each subnet that is used to configure an MCC service must be labelled with special service label that starts with
ipam/SVC-
prefix. Make sure that no subnet has more than one such label.Optional. Add subnet for MetalLB service in your cluster. To designate a Subnet as MetalLB address pool, use label key
ipam/SVC-MetalLB
. Set value of the label to “1”. Set thecluster.sigs.k8s.io/cluster-name
label to the name of the cluster where subnet is used. You may create multiple subnets withipam/SVC-MetalLB
label to define multiple IP address ranges for MetalLB in the cluster.Note
The IP addresses of the MetalLB address pool are not assigned to the interfaces on hosts. This is purely virtual subnet. Make sure that it is not included in the L2 template definitions for your cluster.
Note
When MetalLB address ranges are defined in both cluster specification and specific
Subnet
objects, the resulting MetalLB address pools configuration will contain address ranges from both cluster specification andSubnet
objects.All address ranges for L2 address pools that are defined in both cluster specification and
Subnet
objects are aggregated into a single L2 address pool and sorted as strings.
Optional. Add Ceph ‘public’ subnet.
Set label
ipam/SVC-ceph-public
with value “1” to create a subnet that will be used to configure Ceph ‘public’ network. Ceph will automatically use this subnet for its external connections. A Ceph OSD will look for and bind to an address from this subnet when it is started on a machine. Use this subnet in the L2 template for storage nodes. Assign this subnet to the interface connected to your storage access network.When using this label, set the
cluster.sigs.k8s.io/cluster-name
label to the name of the target cluster during the subnet creation.Optional. Add Ceph ‘replication’ subnet.
Set label
ipam/SVC-ceph-cluster
with value “1” to create a subnet that will be used to configure Ceph ‘replication’ network. Ceph will automatically use this subnet for its internal replication traffic. Use this subnet in the L2 template for storage nodes.When using this label, set the
cluster.sigs.k8s.io/cluster-name
label to the name of the target cluster during the subnet creation.Optional. Add subnet for Kubernetes pods traffic.
Caution
Use of a dedicated network for Kubernetes pods traffic, for external connection to the Kubernetes services exposed by the cluster, and for the Ceph cluster access and replication traffic is available as Technology Preview. Use such configurations for testing and evaluation purposes only. For the Technology Preview feature definition, refer to Technology Preview features.
The following feature is still under development and will be announced in one of the following Container Cloud releases:
Switching Kubernetes API to listen to the specified IP address on the node
Verify that the subnet is successfully created:
kubectl get subnet kaas-mgmt -oyaml
In the system output, verify the
status
fields of thesubnet.yaml
file using the table below.Status fields of the Subnet object¶ Parameter
Description
statusMessage
Contains a short state description and a more detailed one if applicable. The short status values are as follows:
OK
- operational.ERR
- non-operational. This status has a detailed description, for example,ERR: Wrong includeRange for CIDR…
.
cidr
Reflects the actual CIDR, has the same meaning as
spec.cidr
.gateway
Reflects the actual gateway, has the same meaning as
spec.gateway
.nameservers
Reflects the actual name servers, has same meaning as
spec.nameservers
.ranges
Specifies the address ranges that are calculated using the fields from
spec: cidr, includeRanges, excludeRanges, gateway, useWholeCidr
. These ranges are directly used for nodes IP allocation.lastUpdate
Includes the date and time of the latest update of the
Subnet
RC.allocatable
Includes the number of currently available IP addresses that can be allocated for nodes from the subnet.
allocatedIPs
Specifies the list of IPv4 addresses with the corresponding
IPaddr
object IDs that were already allocated from the subnet.capacity
Contains the total number of IP addresses being held by ranges that equals to a sum of the
allocatable
andallocatedIPs
parameters values.versionIpam
Contains thevVersion of the
kaas-ipam
component that made the latest changes to theSubnet
RC.Example of a successfully created subnet:
apiVersion: ipam.mirantis.com/v1alpha1 kind: Subnet metadata: labels: ipam/UID: 6039758f-23ee-40ba-8c0f-61c01b0ac863 kaas.mirantis.com/provider: baremetal kaas.mirantis.com/region: region-one name: kaas-mgmt namespace: default spec: cidr: 10.0.0.0/24 excludeRanges: - 10.0.0.100 - 10.0.0.101-10.0.0.120 gateway: 10.0.0.1 includeRanges: - 10.0.0.50-10.0.0.90 nameservers: - 172.18.176.6 status: allocatable: 38 allocatedIPs: - 10.0.0.50:0b50774f-ffed-11ea-84c7-0242c0a85b02 - 10.0.0.51:1422e651-ffed-11ea-84c7-0242c0a85b02 - 10.0.0.52:1d19912c-ffed-11ea-84c7-0242c0a85b02 capacity: 41 cidr: 10.0.0.0/24 gateway: 10.0.0.1 lastUpdate: "2020-09-26T11:40:44Z" nameservers: - 172.18.176.6 ranges: - 10.0.0.50-10.0.0.90 statusMessage: OK versionIpam: v3.0.999-20200807-130909-44151f8
Now, proceed with creating subnets for your MOSK cluster as described in Create subnets for a MOSK cluster.