Rack¶
TechPreview Available since 2.24.4
This section describes the Rack
resource used in the Container Cloud API.
When you create a bare metal managed cluster with a multi-rack topology,
where Kubernetes masters are distributed across multiple racks
without L2 layer extension between them, the Rack
resource allows you
to configure BGP announcement of the cluster API load balancer address from
each rack.
In this scenario, Rack
objects must be bound to Machine
objects
corresponding to master nodes of the cluster. Each Rack
object describes
the configuration of the BGP daemon (bird) used to announce the cluster API LB
address from a particular master node (or from several nodes in the same rack).
Rack
objects are used for a particular cluster only in conjunction with
the MultiRackCluster
object described in MultiRackCluster.
For demonstration purposes, the Container Cloud Rack
custom resource (CR)
description is split into the following major sections:
For configuration examples, see MultiRackCluster and Rack usage examples.
Rack metadata¶
The Container Cloud Rack
CR metadata contains the following fields:
apiVersion
API version of the object that is
ipam.mirantis.com/v1alpha1
.
kind
Object type that is
Rack
.
metadata
The
metadata
field contains the following subfields:name
Name of the
Rack
object. CorrespondingMachine
objects must have theiripam/RackRef
label value set to the name of theRack
object. This label is required only forMachine
objects of the master nodes that announce the cluster API LB address.
namespace
Container Cloud project (Kubernetes namespace) where the object was created.
labels
Key-value pairs that are attached to the object:
cluster.sigs.k8s.io/cluster-name
Cluster
object name that thisRack
object is applied to.
kaas.mirantis.com/provider
Provider name that is
baremetal
.
kaas.mirantis.com/region
Region name.
Note
The
kaas.mirantis.com/region
label is removed from all Container Cloud objects in 2.26.0 (Cluster releases 17.1.0 and 16.1.0). Therefore, do not add the label starting these releases. On existing clusters updated to these releases, or if manually added, this label will be ignored by Container Cloud.
Warning
Labels and annotations that are not documented in this API Reference are generated automatically by Container Cloud. Do not modify them using the Container Cloud API.
Rack
metadata example:
apiVersion: ipam.mirantis.com/v1alpha1
kind: Rack
metadata:
name: rack-1
namespace: managed-ns
labels:
cluster.sigs.k8s.io/cluster-name: test-cluster
kaas.mirantis.com/provider: baremetal
Corresponding Machine
metadata example:
apiVersion: cluster.k8s.io/v1alpha1
kind: Machine
metadata:
labels:
cluster.sigs.k8s.io/cluster-name: test-cluster
cluster.sigs.k8s.io/control-plane: controlplane
hostlabel.bm.kaas.mirantis.com/controlplane: controlplane
ipam/RackRef: rack-1
kaas.mirantis.com/provider: baremetal
name: managed-master-1-control-efi-6tg52
namespace: managed-ns
Rack spec¶
The spec
field of the Rack
resource describes the desired
state of the object. It contains the following fields:
bgpdConfigTemplate
Optional. Configuration file template that will be used to create configuration file for a BGP daemon on nodes in this rack. If not set, the configuration file template from the corresponding
MultiRackCluster
object is used.
peeringMap
Structure that describes general parameters of BGP peers to be used in the configuration file for a BGP daemon for each network where BGP announcement is used. Also, you can define a separate configuration file template for the BGP daemon for each of those networks. The
peeringMap
structure is as follows:peeringMap: <network-name-a>: peers: - localASN: <localASN-1> neighborASN: <neighborASN-1> neighborIP: <neighborIP-1> password: <password-1> - localASN: <localASN-2> neighborASN: <neighborASN-2> neighborIP: <neighborIP-2> password: <password-2> bgpdConfigTemplate: | <configuration file template for a BGP daemon> ...
<network-name-a>
Name of the network where a BGP daemon should connect to the neighbor BGP peers. By default, it is implied that the same network is used on the node to make connection to the neighbor BGP peers as well as to receive and respond to the traffic directed to the IP address being advertised. In our scenario, the advertised IP address is the cluster API LB IP address.
This network name must be the same as the subnet name used in the L2 template (
l3Layout
section) for the corresponding master node(s).
peers
Optional. List of dictionaries where each dictionary defines configuration parameters for a particular BGP peer. Peer parameters are as follows:
localASN
Optional. Local AS number. If not set, it can be taken from
MultiRackCluster.spec.defaultPeer
or can be hardcoded inbgpdConfigTemplate
.
neighborASN
Optional. Neighbor AS number. If not set, it can be taken from
MultiRackCluster.spec.defaultPeer
or can be hardcoded inbgpdConfigTemplate
.
neighborIP
Mandatory. Neighbor IP address.
password
Optional. Neighbor password. If not set, it can be taken from
MultiRackCluster.spec.defaultPeer
or can be hardcoded inbgpdConfigTemplate
. It is required when MD5 authentication between BGP peers is used.
bgpdConfigTemplate
Optional. Configuration file template that will be used to create the configuration file for the BGP daemon of the
network-name-a
network on a particular node. If not set,Rack.spec.bgpdConfigTemplate
is used.
Configuration example:
Since Cluster releases 17.1.0 and 16.1.0 for bird
v2.x
spec:
bgpdConfigTemplate: |
protocol device {
}
#
protocol direct {
interface "lo";
ipv4;
}
#
protocol kernel {
ipv4 {
export all;
};
}
#
protocol bgp bgp_lcm {
local port 1179 as {{.LocalASN}};
neighbor {{.NeighborIP}} as {{.NeighborASN}};
ipv4 {
import none;
export filter {
if dest = RTD_UNREACHABLE then {
reject;
}
accept;
};
};
}
peeringMap:
lcm-rack1:
peers:
- localASN: 65050
neighborASN: 65011
neighborIP: 10.77.31.1
Before Cluster releases 17.1.0 and 16.1.0 for bird
v1.x
spec:
bgpdConfigTemplate: |
listen bgp port 1179;
protocol device {
}
#
protocol direct {
interface "lo";
}
#
protocol kernel {
export all;
}
#
protocol bgp bgp_lcm {
local as {{.LocalASN}};
neighbor {{.NeighborIP}} as {{.NeighborASN}};
import all;
export filter {
if dest = RTD_UNREACHABLE then {
reject;
}
accept;
};
}
peeringMap:
lcm-rack1:
peers:
- localASN: 65050
neighborASN: 65011
neighborIP: 10.77.31.1
Rack status¶
The status
field of the Rack
resource reflects the actual state
of the Rack
object and contains the following fields:
state
Since 2.23.0Message that reflects the current status of the resource. The list of possible values includes the following:
OK
- object is operational.ERR
- object is non-operational. This status has a detailed description in themessages
list.TERM
- object was deleted and is terminating.
messages
Since 2.23.0List of error or warning messages if the object state is
ERR
.
objCreated
Date, time, and IPAM version of the resource creation.
objStatusUpdated
Date, time, and IPAM version of the last update of the
status
field in the resource.
objUpdated
Date, time, and IPAM version of the last resource update.
Configuration example:
status:
checksums:
annotations: sha256:cd4b751d9773eacbfd5493712db0cbebd6df0762156aefa502d65a9d5e8af31d
labels: sha256:fc2612d12253443955e1bf929f437245d304b483974ff02a165bc5c78363f739
spec: sha256:8f0223b1eefb6a9cd583905a25822fd83ac544e62e1dfef26ee798834ef4c0c1
objCreated: 2023-08-11T12:25:21.00000Z by v6.5.999-20230810-155553-2497818
objStatusUpdated: 2023-08-11T12:33:00.92163Z by v6.5.999-20230810-155553-2497818
objUpdated: 2023-08-11T12:32:59.11951Z by v6.5.999-20230810-155553-2497818
state: OK