Increase BGP max_prefix

All Equinix Metal projects have the default value of Border Gateway Protocol (BGP) max_prefix parameter set to 10. The BGP max_prefix parameter is the maximum number of route filters allowed per server. It means that when the Equinix server receives too many prefixes from a peer and the maximum prefix limit exceeds, the peering session disables or is brought down and the deployed services may become inaccessible.

By default, the Container Cloud uses:

  • 3 prefixes for the management clusters services (/29, /31, /32), 11 Elastic IPs in total

  • 1 prefix for the regional cluster services (/29), 8 Elastic IPs in total

  • 2 prefixes for managed clusters (/30, /31), 6 Elastic IPs in total

The default value of 10 for the BGP max_prefix parameter should be enough if no additional services with the type LoadBalancer are running on the cluster. Otherwise, the value should be increased correspondingly.

When a new Kubernetes service with the type LoadBalancer is created in the cluster, the provider requests a new Elastic IP block (/32). Once the IP is provided, it is automatically assigned to the service and added to the BGP prefixes list. Once the maximum number of prefixes is reached, the peering session disables and all cluster services become inaccessible. Thus, increase the BGP max_prefix for the project.

Example of the BGP max_prefix calculation:

  • The managed cluster was deployed in the project with the default BGP max_prefix parameter set to 10

  • Two BGP prefixes are in use by the Container Cloud workloads

  • Nine additional services with the LoadBalancer type are expected to be deployed on the managed cluster

In this case, set the value of BGP max_prefix to at least 11 to ensure that the limit is not reached:

  • /30 and /31 prefixes for the Container Cloud workloads

  • 9 x /32 prefixes for user workloads

Note

Set the BGP max_prefix parameter to the maximum value of prefixes required by each cluster in the project. For example, if one cluster requires 11 prefixes and another one requires only 2, set at least 11 as the BGP max_prefix value.

To verify the BGP max_prefix value:

  1. Set the token variable to your project token.

    To obtain the token using the Equinix Metal console, navigate to Project Settings > Project API Keys > Add New Key.

  2. Set the project variable to your project ID.

    To obtain the project ID using the Equinix Metal console, navigate to Project Settings > General > PROJECT ID.

  3. Verify the value of the BGP max_prefix parameter:

    curl -sS -H "X-Auth-Token: ${token}" "https://api.equinix.com/metal/v1/projects/${project}/bgp-config" | jq .max_prefix

    In the system output, if the value is less than required, contact the Equinix Metal support to increase this parameter to the value calculated using the example provided above.