Use NVLink topology-aware scheduling for multi-GPU instances

TechPreview

NVLink topology-aware scheduling lets you launch an instance with several physical GPUs and have the cloud pick devices that share a high-bandwidth NVLink or NVSwitch domain. Typical use cases include virtual machines for multi-GPU training and inference that need GPU-to-GPU bandwidth inside one host.

Without this capability, cloud operators expose a separate flavor for every valid pair or quad of cards. Those flavors look identical to you, but only some of them still have free GPUs. Finding a working combination becomes a matter of trial and error, and a failed request often surfaces later as an instance in ERROR state rather than as an immediate inventory error.

With topology-aware scheduling configured, the cloud operator publishes a small set of flavors by GPU count and type, for example one, two, four, or eight GPUs of a given model. You choose the flavor that matches the size of your workload. MOSK selects only GPUs from the same documented NVLink group. If inventory cannot satisfy that constraint, scheduling fails. There is no fallback to GPUs from different domains.

Before using the feature, confirm with your cloud administrator that the capability is enabled for your environment and ask which flavor names to use. For architecture and limitations, see NVLink topology-aware scheduling of multi-GPU instances.

Launch an instance

Use the flavor your administrator documented for the GPU count you need. Launch the instance as usual with the OpenStack CLI, Heat, or another client that accepts a flavor. For example, using the OpenStack CLI:

openstack server create \
    --image <IMAGE> \
    --flavor <GPU-FLAVOR> \
    --network <NETWORK> \
    --key-name <KEY> \
    <SERVER-NAME>

Replace <GPU-FLAVOR> with the published flavor for one, two, four, or eight GPUs. Do not mix these flavors with virtual GPU (vGPU or MIG) flavors. NVLink-aware scheduling applies to PCI passthrough of whole GPUs, not to partitioned virtual GPUs.

When scheduling succeeds, the instance becomes ACTIVE and the guest sees the requested number of GPUs. Install the appropriate NVIDIA guest driver for PCI passthrough as required by your image.

Failures to expect

If no remaining GPUs share a group of the size you requested, scheduling fails. The instance typically enters ERROR with a no-valid-host fault rather than booting on a mismatched set of cards.

A host tool such as nvidia-smi can still show unused GPUs when Placement cannot use them together. Common cases:

• The free GPUs do not sit in the same NVLink group of two or four. An eight-GPU request uses all GPUs on the host.

• NUMA pinning on the flavor (dedicated CPUs or an explicit NUMA extra spec) conflicts with the NVLink group the scheduler would need. Ask the administrator which GPU flavors also constrain NUMA.

PCI-passthrough GPUs follow the same live-migration and hot-plug limits as other passthrough devices. You cannot live-migrate the instance or attach and detach these GPUs on a running instance.

If a launch fails, retry with a smaller GPU count if your workload allows it, or ask the administrator whether inventory remains in a single NVLink group.

See also

NVLink topology-aware scheduling of multi-GPU instances