Design details

To improve isolation, each cluster is placed in its own private network (VLAN). All other connectivity, including access to the Equinix Metal internal network, is disabled on all nodes.

The following diagram illustrates a high-level overview of the architecture. It covers the Container Cloud deployment across multiple Metros, marked as A and B on the diagram.

../../_images/equinix-v2-overview.png

Main services configuration

The main element in the private networking architecture is the infrastructure-level IP routing provided by routers that are connected to the VLAN of each cluster by enabling communication between clusters.

Another critical service that routers provide is DHCP relay. It forwards DHCP/PXE requests from managed cluster nodes to the Ironic DHCP server in the management cluster.

Example configuration:

  • Metro A has 2 VLANS:

    • VLAN 1001 with the IP range 192.168.0.0/24 connected to the router interface bond0.1001 with the IP 192.168.0.1 for the management cluster

    • VLAN 1002 with IP range 192.168.1.0/24 connected to the router interface bond0.1002 with IP 192.168.1.1 for the managed cluster

  • The IP address of the Ironic DHCP server is 192.168.0.50

With these settings, the router should have the following configuration:

  • Forwarding between the bond0.1001 and bond0.1002 interfaces

  • Route for 192.168.0.0/24 via the bond0.1001 interface

  • Route for 192.168.1.0/24 via the bond0.1002 interface

  • DHCP relay on both bond0.1001 and bond0.1002 interfaces forwarding DHCP requests to 192.168.0.50

  • One of the following services:

    • SNAT for traffic from the 192.168.0.0/24 and 192.168.1.0/24 networks to the Internet or the proxy service

    • Proxy service for the Mirantis CDN and telemetry servers running on the router itself

During Container Cloud deployment, the seed node will have to:

  • Be deployed in Metro A

  • Be directly connected to VLAN 1001

  • Have access to the proxy service or the Internet

Managed clusters in different Metros

If users need to deploy clusters in different Metros, every Metro requires a separate router and VLAN configuration.

For example, Metro B has a separate router and 2 VLANS:

  • VLAN 2001 with the IP range 192.168.16.0/24 connected to the router interface bond0.2001 with the IP 192.168.16.1 for the management cluster

  • VLAN 2002 with the IP range 192.168.17.0/24 connected to the router interface bond0.2002 with the IP 192.168.17.1 for the managed cluster

With this configuration, to deploy managed clusters using the management cluster from Metro A, the router in Metro B should have the following configuration:

  • The VXLAN interface (or any other tunnel) vxlan1, for example, with the IP address 192.168.255.2 and the remote address of the router in Metro A

  • Forwarding between the bond0.2001, bond0.2002, and vxlan1 interfaces

  • Route for 192.168.16.0/24 via the bond0.1001 interface

  • Route for 192.168.17.0/24 via the bond0.1002 interface

  • Route for 192.168.0.0/20 via 192.168.255.1 that is the VXLAN address of the router in Metro A

  • DHCP relay on both bond0.1001 and bond0.1002 interfaces forwarding DHCP requests to 192.168.0.50

The router in Metro A requires additional configuration for:

  • The VXLAN interface (or any other tunnel) vxlan1, for example, with the IP address 192.168.255.1 and the remote address of the router in Metro B

  • Route for 192.168.16.0/20 via 192.168.255.2 that is the VXLAN address of the router in Metro B

Additional regional cluster

You can deploy a separate regional cluster in Metro B to reduce traffic between Metros. For example, if you use VLAN 2001 for the regional cluster, the router in Metro B requires additional configuration for:

  • DHCP relay on both bond0.2001 and bond0.2002 interfaces forwarding DHCP requests to 192.168.16.50 that is the IP address of the Ironic DHCP server in the regional cluster

  • SNAT or proxy service to allow the regional cluster access to the Mirantis CDN and telemetry

During regional cluster deployment, the seed node for the regional cluster will have to:

  • Be deployed in Metro B

  • Be directly connected to VLAN 2001

  • Have access to the proxy service or the Internet

The following diagram illustrates the Container Cloud deployment with an additional regional cluster:

../../_images/equinix-v2-regional.png

Note

For an example of Terraform templates and Ansible playbooks to use for deployment and configuration of all components described above, see Container Cloud on Equinix Metal templates.