MetalLBConfig

TechPreview since 2.21.0 and 2.21.1 for MOSK 22.5 GA since 2.24.0 for management and regional clusters GA since 2.25.0 for managed clusters

This section describes the MetalLBConfig custom resource used in the Container Cloud API that contains the MetalLB configuration objects for a particular cluster.

For demonstration purposes, the Container Cloud MetalLBConfig custom resource description is split into the following major sections:

The Container Cloud API also uses the third-party open source MetalLB API. For details, see MetalLB objects.

MetalLBConfig metadata

The Container Cloud MetalLBConfig CR contains the following fields:

  • apiVersion

    API version of the object that is kaas.mirantis.com/v1alpha1.

  • kind

    Object type that is MetalLBConfig.

The metadata object field of the MetalLBConfig resource contains the following fields:

  • name

    Name of the MetalLBConfig object.

  • namespace

    Project in which the object was created. Must match the project name of the target cluster.

  • labels

    Key-value pairs attached to the object. Mandatory labels:

    • kaas.mirantis.com/provider

      Provider type that is baremetal.

    • kaas.mirantis.com/region

      Region name that matches the region name of the target cluster.

      Note

      The kaas.mirantis.com/region label is removed from all Container Cloud objects in 2.26.0 (Cluster releases 17.1.0 and 16.1.0). Therefore, do not add the label starting these releases. On existing clusters updated to these releases, or if manually added, this label will be ignored by Container Cloud.

    • cluster.sigs.k8s.io/cluster-name

      Name of the cluster that the MetalLB configuration must apply to.

    Warning

    Labels and annotations that are not documented in this API Reference are generated automatically by Container Cloud. Do not modify them using the Container Cloud API.

Configuration example:

apiVersion: kaas.mirantis.com/v1alpha1
kind: MetalLBConfig
metadata:
  name: metallb-demo
  namespace: test-ns
  labels:
    kaas.mirantis.com/provider: baremetal
    cluster.sigs.k8s.io/cluster-name: test-cluster

MetalLBConfig spec

The spec field of the MetalLBConfig object represents the MetalLBConfigSpec subresource that contains the description of MetalLB configuration objects. These objects are created in the target cluster during its deployment.

The spec field contains the following optional fields:

  • addressPools

    Removed in Container Cloud 2.27.0 (Cluster releases 17.2.0 and 16.2.0), deprecated in 2.26.0 (Cluster releases 17.2.0 and 16.2.0).

    List of MetalLBAddressPool objects to create MetalLB AddressPool objects.

  • bfdProfiles

    List of MetalLBBFDProfile objects to create MetalLB BFDProfile objects.

  • bgpAdvertisements

    List of MetalLBBGPAdvertisement objects to create MetalLB BGPAdvertisement objects.

  • bgpPeers

    List of MetalLBBGPPeer objects to create MetalLB BGPPeer objects.

  • communities

    List of MetalLBCommunity objects to create MetalLB Community objects.

  • ipAddressPools

    List of MetalLBIPAddressPool objects to create MetalLB IPAddressPool objects.

  • l2Advertisements

    List of MetalLBL2Advertisement objects to create MetalLB L2Advertisement objects.

    The l2Advertisements object allows defining interfaces to optimize the announcement. When you use the interfaces selector, LB addresses are announced only on selected host interfaces.

    Mirantis recommends using the interfaces selector if nodes use separate host networks for different types of traffic. The pros of such configuration are as follows: less spam on other interfaces and networks and limited chances to reach IP addresses of load-balanced services from irrelevant interfaces and networks.

    Caution

    Interface names in the interfaces list must match those on the corresponding nodes.

  • templateName

    Unsupported since 2.28.0 (17.3.0 and 16.3.0). Available since 2.24.0 (14.0.0). For details, see MOSK Deprecation Notes: MetalLBConfigTemplate resource management.

    Name of the MetalLBConfigTemplate object used as a source of MetalLB configuration objects. Mutually exclusive with the fields listed below that will be part of the MetalLBConfigTemplate object. For details, see MetalLBConfigTemplate.

    Before Cluster releases 17.2.0 and 16.2.0, MetalLBConfigTemplate is the default configuration method for MetalLB on bare metal deployments. Since Cluster releases 17.2.0 and 16.2.0, use the MetalLBConfig object instead.

    Caution

    For MKE clusters that are part of MOSK infrastructure, the feature support will become available in one of the following Container Cloud releases.

    Caution

    For managed clusters, this field is available as Technology Preview since Container Cloud 2.24.0, is generally available since 2.25.0, and is deprecated since 2.27.0.


The objects listed in the spec field of the MetalLBConfig object, such as MetalLBIPAddressPool, MetalLBL2Advertisement, and so on, are used as templates for the MetalLB objects that will be created in the target cluster. Each of these objects has the following structure:

  • labels

    Optional. Key-value pairs attached to the metallb.io/<objectName> object as metadata.labels.

  • name

    Name of the metallb.io/<objectName> object.

  • spec

    Contents of the spec section of the metallb.io/<objectName> object. The spec field has the metallb.io/<objectName>Spec type. For details, see MetalLB objects.

For example, MetalLBIPAddressPool is a template for the metallb.io/IPAddressPool object and has the following structure:

  • labels

    Optional. Key-value pairs attached to the metallb.io/IPAddressPool object as metadata.labels.

  • name

    Name of the metallb.io/IPAddressPool object.

  • spec

    Contents of spec section of the metallb.io/IPAddressPool object. The spec has the metallb.io/IPAddressPoolSpec type.

MetalLB objects

Container Cloud supports the following MetalLB object types of the metallb.io API group:

  • IPAddressPool

  • Community

  • L2Advertisement

  • BFDProfile

  • BGPAdvertisement

  • BGPPeer

As of v1beta1 and v1beta2 API versions, metadata of MetalLB objects has a standard format with no specific fields or labels defined for any particular object:

  • apiVersion

    API version of the object that can be metallb.io/v1beta1 or metallb.io/v1beta2.

  • kind

    Object type that is one of the metallb.io types listed above. For example, IPAddressPool.

  • metadata

    Object metadata that contains the following subfields:

    • name

      Name of the object.

    • namespace

      Namespace where the MetalLB components are located. It matches metallb-system in Container Cloud.

    • labels

      Optional. Key-value pairs that are attached to the object. It can be an arbitrary set of labels. No special labels are defined as of v1beta1 and v1beta2 API versions.

The MetalLBConfig object contains spec sections of the metallb.io/<objectName> objects that have the metallb.io/<objectName>Spec type. For metallb.io/<objectName> and metallb.io/<objectName>Spec types definitions, refer to the official MetalLB documentation:

Note

Before Container Cloud 2.27.0 (Cluster releases 17.2.0 and 16.2.0), metallb.io/<objectName> objects v0.13.9 are supported.

The l2Advertisements object allows defining interfaces to optimize the announcement. When you use the interfaces selector, LB addresses are announced only on selected host interfaces. Mirantis recommends this configuration if nodes use separate host networks for different types of traffic. The pros of such configuration are as follows: less spam on other interfaces and networks, limited chances to reach services LB addresses from irrelevant interfaces and networks.

Configuration example:

l2Advertisements: |
  - name: management-lcm
    spec:
      ipAddressPools:
        - default
      interfaces:
        # LB addresses from the "default" address pool will be announced
        # on the "k8s-lcm" interface
        - k8s-lcm

Caution

Interface names in the interfaces list must match those on the corresponding nodes.

MetalLBConfig status

Available since 2.24.0 for management clusters

Caution

For managed clusters, this field is available as Technology Preview and is generally available since Container Cloud 2.25.0.

Caution

For MKE clusters that are part of MOSK infrastructure, the feature support will become available in one of the following Container Cloud releases.

The status field describes the actual state of the object. It contains the following fields:

  • bootstrapMode Only in 2.24.0

    Field that appears only during a management cluster bootstrap as true and is used internally for bootstrap. Once deployment completes, the value is moved to false and is excluded from the status output.

  • objects

    Description of MetalLB objects that is used to create MetalLB native objects in the target cluster.

    The format of underlying objects is the same as for those in the spec field, except templateName, which is obsolete since Container Cloud 2.28.0 (Cluster releases 17.3.0 and 16.3.0) and which is not present in this field. The objects contents are rendered from the following locations, with possible modifications for the bootstrap cluster:

    • Since Container Cloud 2.28.0 (Cluster releases 17.3.0 and 16.3.0), MetalLBConfig.spec

    • Before Container Cloud 2.28.0 (Cluster releases 17.2.0, 16.2.0, or earlier):

      • MetalLBConfigTemplate.status of the corresponding template if MetalLBConfig.spec.templateName is defined

      • MetalLBConfig.spec if MetalLBConfig.spec.templateName is not defined

  • propagateResult

    Result of objects propagation. During objects propagation, native MetalLB objects of the target cluster are created and updated according to the description of the objects present in the status.objects field.

    This field contains the following information:

    • message

      Text message that describes the result of the last attempt of objects propagation. Contains an error message if the last attempt was unsuccessful.

    • success

      Result of the last attempt of objects propagation. Boolean.

    • time

      Timestamp of the last attempt of objects propagation. For example, 2023-07-04T00:30:36Z.

    If the objects propagation was successful, the MetalLB objects of the target cluster match the ones present in the status.objects field.

  • updateResult

    Status of the MetalLB objects update. Has the same format of subfields that in propagateResult described above.

    During objects update, the status.objects contents are rendered as described in the objects field definition above.

    If the objects update was successful, the MetalLB objects description present in status.objects is rendered successfully and up to date. This description is used to update MetalLB objects in the target cluster. If the objects update was not successful, MetalLB objects will not be propagated to the target cluster.

MetalLB configuration examples

Example of configuration template for using L2 announcements:

apiVersion: kaas.mirantis.com/v1alpha1
kind: MetalLBConfig
metadata:
  labels:
    cluster.sigs.k8s.io/cluster-name: managed-cluster
    kaas.mirantis.com/provider: baremetal
  name: managed-l2
  namespace: managed-ns
spec:
  ipAddressPools:
    - name: services
      spec:
        addresses:
          - 10.100.91.151-10.100.91.170
        autoAssign: true
        avoidBuggyIPs: false
  l2Advertisements:
    - name: services
      spec:
        ipAddressPools:
        - services

Example of configuration extract for using the interfaces selector, which enables announcement of LB addresses only on selected host interfaces:

l2Advertisements:
  - name: services
    spec:
      ipAddressPools:
      - default
      interfaces:
      - k8s-lcm

Caution

Interface names in the interfaces list must match the ones on the corresponding nodes.

After the object is created and processed by the MetalLB Controller, the status field is added. For example:

status:
  objects:
    ipAddressPools:
    - name: services
      spec:
        addresses:
        - 10.100.100.151-10.100.100.170
        autoAssign: true
        avoidBuggyIPs: false
    l2Advertisements:
      - name: services
        spec:
          ipAddressPools:
          - services
  propagateResult:
    message: Objects were successfully updated
    success: true
    time: "2023-07-04T14:31:40Z"
  updateResult:
    message: Objects were successfully read from MetalLB configuration specification
    success: true
    time: "2023-07-04T14:31:39Z"

Example of native MetalLB objects to be created in the managed-ns/managed-cluster cluster during deployment:

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: services
  namespace: metallb-system
spec:
  addresses:
  - 10.100.91.151-10.100.91.170
  autoAssign: true
  avoidBuggyIPs: false
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: services
  namespace: metallb-system
spec:
  ipAddressPools:
  - services

Example of configuration template for using BGP announcements:

apiVersion: kaas.mirantis.com/v1alpha1
kind: MetalLBConfig
metadata:
  labels:
    cluster.sigs.k8s.io/cluster-name: managed-cluster
    kaas.mirantis.com/provider: baremetal
  name: managed-bgp
  namespace: managed-ns
spec:
  bgpPeers:
    - name: bgp-peer-rack1
      spec:
        peerAddress: 10.0.41.1
        peerASN: 65013
        myASN: 65099
        nodeSelectors:
          - matchLabels:
              rack-id: rack1
    - name: bgp-peer-rack2
      spec:
        peerAddress: 10.0.42.1
        peerASN: 65023
        myASN: 65099
        nodeSelectors:
          - matchLabels:
              rack-id: rack2
    - name: bgp-peer-rack3
      spec:
        peerAddress: 10.0.43.1
        peerASN: 65033
        myASN: 65099
        nodeSelectors:
          - matchLabels:
              rack-id: rack3
  ipAddressPools:
    - name: services
      spec:
        addresses:
          - 10.100.191.151-10.100.191.170
        autoAssign: true
        avoidBuggyIPs: false
  bgpAdvertisements:
    - name: services
      spec:
        ipAddressPools:
        - services

See also