Create subnets¶

Before creating an L2 template, ensure that you have the required subnets that can be used in the L2 template to allocate IP addresses for the managed cluster nodes. Where required, create a number of subnets for a particular project using the Subnet CR. A subnet has three logical scopes:

global - CR uses the default namespace. A subnet can be used for any cluster located in any project.
namespaced - CR uses the namespace that corresponds to a particular project where managed clusters are located. A subnet can be used for any cluster located in the same project.
cluster - CR uses the namespace where the referenced cluster is located. A subnet is only accessible to the cluster that L2Template.spec.clusterRef refers to. The Subnet objects with the cluster scope will be created for every new cluster.

You can have subnets with the same name in different projects. In this case, the subnet that has the same project as the cluster will be used. One L2 template may often reference several subnets, those subnets may have different scopes in this case.

The IP address objects (IPaddr CR) that are allocated from subnets always have the same project as their corresponding IpamHost objects, regardless of the subnet scope.

You can create subnets using either the Container Cloud web UI or CLI.

To create subnets for a managed cluster using web UI:

Log in to the Container Cloud web UI with the operator permissions.
Switch to the required project using the Switch Project action icon located on top of the main left-side navigation panel.
In the Clusters tab, click the required cluster and scroll down to the Subnets section.
Click Add Subnet.
Fill out the Add new subnet form as required:
- Subnet Name
  Subnet name.
- CIDR
  A valid IPv4 CIDR, for example, 10.11.0.0/24.
- Include Ranges ^Optional
  A list of IP address ranges within the given CIDR that should be used in the allocation of IPs for nodes. The gateway, network, broadcast, and DNS addresses will be excluded (protected) automatically if they intersect with one of the range. The IPs outside the given ranges will not be used in the allocation. Each element of the list can be either an interval 10.11.0.5-10.11.0.70 or a single address 10.11.0.77. The includeRanges parameter is mutually exclusive with excludeRanges.
- Exclude Ranges ^Optional
  A list of IP address ranges within the given CIDR that should not be used in the allocation of IPs for nodes. The IPs within the given CIDR but outside the given ranges will be used in the allocation. The gateway, network, broadcast, and DNS addresses will be excluded (protected) automatically if they are included in the CIDR. Each element of the list can be either an interval 10.11.0.5-10.11.0.70 or a single address 10.11.0.77. The excludeRanges parameter is mutually exclusive with includeRanges.
- Gateway ^Optional
  A valid gateway address, for example, 10.11.0.9.
Click Create.

To create subnets for a managed cluster using CLI:

Log in to a local machine where your management cluster kubeconfig is located and where kubectl is installed.

Note

The management cluster kubeconfig is created during the last stage of the management cluster bootstrap.

Create the subnet.yaml file with a number of global or namespaced subnets depending on the configuration of your cluster:

kubectl --kubeconfig <pathToManagementClusterKubeconfig> apply -f <SubnetFileName.yaml>

Note

In the command above and in the steps below, substitute the parameters enclosed in angle brackets with the corresponding values.

Example of a subnet.yaml file:

apiVersion: ipam.mirantis.com/v1alpha1
kind: Subnet
metadata:
  name: demo
  namespace: demo-namespace
  labels:
    kaas.mirantis.com/provider: baremetal
    kaas.mirantis.com/region: region-one
spec:
  cidr: 10.11.0.0/24
  gateway: 10.11.0.9
  includeRanges:
  - 10.11.0.5-10.11.0.70
  nameservers:
  - 172.18.176.6

Specification fields of the Subnet object¶
Parameter	Description
`cidr` (singular)	A valid IPv4 CIDR, for example, 10.11.0.0/24.
`includeRanges` (list)	A list of IP address ranges within the given CIDR that should be used in the allocation of IPs for nodes. The gateway, network, broadcast, and DNS addresses will be excluded (protected) automatically if they intersect with one of the range. The IPs outside the given ranges will not be used in the allocation. Each element of the list can be either an interval `10.11.0.5-10.11.0.70` or a single address `10.11.0.77`. The `includeRanges` parameter is mutually exclusive with `excludeRanges`.
`excludeRanges` (list)	A list of IP address ranges within the given CIDR that should not be used in the allocation of IPs for nodes. The IPs within the given CIDR but outside the given ranges will be used in the allocation. The gateway, network, broadcast, and DNS addresses will be excluded (protected) automatically if they are included in the CIDR. Each element of the list can be either an interval `10.11.0.5-10.11.0.70` or a single address `10.11.0.77`. The `excludeRanges` parameter is mutually exclusive with `includeRanges`.
`useWholeCidr` (boolean)	If set to `true`, the subnet address (10.11.0.0 in the example above) and the broadcast address (10.11.0.255 in the example above) are included into the address allocation for nodes. Otherwise, (`false` by default), the subnet address and broadcast address will be excluded from the address allocation.
`gateway` (singular)	A valid gateway address, for example, 10.11.0.9.
`nameservers` (list)	A list of the IP addresses of name servers. Each element of the list is a single address, for example, 172.18.176.6.

Caution

The subnet for the PXE network is automatically created during deployment and must contain the ipam/DefaultSubnet: "1" label. Each bare metal region must have only one subnet with this label.

Caution

You may use different subnets to allocate IP addresses to different Container Cloud components in your cluster. For details, see the optional steps below.

Add a label with the ipam/SVC- prefix to each subnet that is used to configure a Container Cloud service. Make sure that each subnet has only one such label.

Optional. Add a subnet for the MetalLB service.
- To designate a subnet as MetalLB address pool, use the ipam/SVC-MetalLB label key. Set value of the label to "1".
- Set the cluster.sigs.k8s.io/cluster-name label to the name of the cluster where this subnet is used.
- You may create multiple subnets with the ipam/SVC-MetalLB label to define multiple IP address pools for MetalLB in the cluster.
Caution

The IP addresses of the MetalLB address pool are not assigned to the interfaces on hosts. This subnet is virtual. Make sure that it is not included in the L2 template definitions for your cluster.
Optional. Add a Ceph public subnet.
- Set the ipam/SVC-ceph-public label with the value "1" to create a subnet that will be used to configure the Ceph public network.
- Use this subnet in the L2 template for storage nodes.
- Assign this subnet to the interface connected to your storage access network.
- Set the cluster.sigs.k8s.io/cluster-name label to the name of the target cluster during this subnet creation.
- Ceph will automatically use this subnet for its external connections.
- A Ceph OSD will look for and bind to an address from this subnet when it is started on a machine.
Optional. Add a Ceph replication subnet.
- Set the ipam/SVC-ceph-cluster label with the value "1" to create a subnet that will be used to configure the Ceph replication network.
- Set the cluster.sigs.k8s.io/cluster-name label to the name of the target cluster during this subnet creation.
- Use this subnet in the L2 template for storage nodes.
- Ceph will automatically use this subnet for its internal replication traffic.
Optional. Add a subnet for Kubernetes pods traffic.
Caution

Use of a dedicated network for Kubernetes pods traffic, for external connection to the Kubernetes services exposed by the cluster, and for the Ceph cluster access and replication traffic is available as Technology Preview. Use such configurations for testing and evaluation purposes only. For the Technology Preview feature definition, refer to Technology Preview support scope.

The following feature is still under development and will be announced in one of the following Container Cloud releases:
- Switching Kubernetes API to listen to the specified IP address on the node

Verify that the subnet is successfully created:

kubectl get subnet kaas-mgmt -oyaml

In the system output, verify the status fields of the subnet.yaml file using the table below.

Status fields of the Subnet object¶
Parameter	Description
`statusMessage`	Contains a short state description and a more detailed one if applicable. The short status values are as follows: `OK` - operational. `ERR` - non-operational. This status has a detailed description, for example, `ERR: Wrong includeRange for CIDR…`.
`cidr`	Reflects the actual CIDR, has the same meaning as `spec.cidr`.
`gateway`	Reflects the actual gateway, has the same meaning as `spec.gateway`.
`nameservers`	Reflects the actual name servers, has same meaning as `spec.nameservers`.
`ranges`	Specifies the address ranges that are calculated using the fields from `spec: cidr, includeRanges, excludeRanges, gateway, useWholeCidr`. These ranges are directly used for nodes IP allocation.
`lastUpdate`	Includes the date and time of the latest update of the `Subnet` RC.
`allocatable`	Includes the number of currently available IP addresses that can be allocated for nodes from the subnet.
`allocatedIPs`	Specifies the list of IPv4 addresses with the corresponding `IPaddr` object IDs that were already allocated from the subnet.
`capacity`	Contains the total number of IP addresses being held by ranges that equals to a sum of the `allocatable` and `allocatedIPs` parameters values.
`versionIpam`	Contains thevVersion of the `kaas-ipam` component that made the latest changes to the `Subnet` RC.

Example of a successfully created subnet:

apiVersion: ipam.mirantis.com/v1alpha1
kind: Subnet
metadata:
  labels:
    ipam/UID: 6039758f-23ee-40ba-8c0f-61c01b0ac863
    kaas.mirantis.com/provider: baremetal
    kaas.mirantis.com/region: region-one
  name: kaas-mgmt
  namespace: default
spec:
  cidr: 10.0.0.0/24
  excludeRanges:
  - 10.0.0.100
  - 10.0.0.101-10.0.0.120
  gateway: 10.0.0.1
  includeRanges:
  - 10.0.0.50-10.0.0.90
  nameservers:
  - 172.18.176.6
status:
  allocatable: 38
  allocatedIPs:
  - 10.0.0.50:0b50774f-ffed-11ea-84c7-0242c0a85b02
  - 10.0.0.51:1422e651-ffed-11ea-84c7-0242c0a85b02
  - 10.0.0.52:1d19912c-ffed-11ea-84c7-0242c0a85b02
  capacity: 41
  cidr: 10.0.0.0/24
  gateway: 10.0.0.1
  lastUpdate: "2020-09-26T11:40:44Z"
  nameservers:
  - 172.18.176.6
  ranges:
  - 10.0.0.50-10.0.0.90
  statusMessage: OK
  versionIpam: v3.0.999-20200807-130909-44151f8

Proceed to creating an L2 template for one or multiple managed clusters as described in Create L2 templates.