Compose multiple accelerator vendors in the serving stack: Provision Vultr bare metal servers into k3s inference clusters - #434
Draft
haarchri wants to merge 2 commits into
Draft
Conversation
Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
haarchri
force-pushed
the
feature/baremetal-amd
branch
from
September 9, 2026 13:28
06e3317 to
03d64b9
Compare
Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of your changes
Design Validation: #432
Vultrs biggest GPUs (8x MI355X) are bare metal plans with no managed Kubernetes in front, and Modelplane could neither provision metal nor install an AMD GPU stack.
A new K3sCluster XR turns machines into a cluster: it installs a k3s server on the control plane machine and joins workers as agents over SSH via provider-k3s, publishing the kubeconfig through the same
status.secretscontract as the other cluster XRs. AVultrBaremetalClusterXR provisions the servers on top, one CPU-only management server plus the GPU pools, SSH key registration, and cloud-init opening the k3s ports (Vultrs Ubuntu images firewall everything but SSH) and composes theK3sClusterfrom their IPs. InferenceCluster gains aVultrBaremetalsource; pools are fixed size and only the Standard stack is supported.The serving stack implements the multi-accelerator design (#432): the
VultrBaremetalcomponent list carries both the AMD GPU operator (DRA mode) and the NVIDIA operator + DRA driver, vendor-tagged and filtered by the accelerators the InferenceClasses name. Unsupported class/cloud pairings fail early withUnsupportedDevicesconditions.Validated live on Vultr with CPU-only plans: servers provisioned, k3s v1.34.11 installed over SSH, the worker joined with pool labels and vendor taint, kubeconfig flowed to the serving stack install. The management server is a single k3s server and the data plane rides public IPs
ToDo:
Fixes #
I have:
nix flake check(or./nix.sh flake check) and made sure it passes.git commit -s.Tested: