Skip to content

Compose multiple accelerator vendors in the serving stack: Provision Vultr bare metal servers into k3s inference clusters #434

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
haarchri wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Compose multiple accelerator vendors in the serving stack: Provision Vultr bare metal servers into k3s inference clusters #434

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
03d64b9
feat(amd): add vultr bare metal servers into k3s inference clusters
haarchri Sep 8, 2026
86b5fc5
lint(py): fix lint issue
haarchri Sep 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 38 additions & 1 deletion apis/inferenceclasses/definition.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,12 @@ spec:
message: spec.provisioning.nebius is required when spec.provisioning.provider is Nebius.
- rule: "self.provider != 'Vultr' || has(self.vultr)"
message: spec.provisioning.vultr is required when spec.provisioning.provider is Vultr.
- rule: "self.provider != 'VultrBaremetal' || has(self.vultrBaremetal)"
message: spec.provisioning.vultrBaremetal is required when spec.provisioning.provider is VultrBaremetal.
properties:
provider:
type: string
enum: [GKE, EKS, AKS, Nebius, Vultr]
enum: [GKE, EKS, AKS, Nebius, Vultr, VultrBaremetal]
gke:
type: object
required: [machineType, accelerator]
Expand Down Expand Up @@ -242,6 +244,41 @@ spec:
type: integer
minimum: 1
maximum: 16
vultrBaremetal:
type: object
required: [plan, accelerator]
properties:
plan:
type: string
description: >-
Vultr bare metal plan ID (e.g.
vbm-256c-3072gb-8-mi355x-gpu). The plan
determines the GPU model and count; the
accelerator block below is informational.
minLength: 1
maxLength: 63
accelerator:
type: object
description: >-
GPU accelerator the plan carries. Provisioning
input only: the scheduler matches against
spec.devices, not this block. The type's vendor
prefix (amd-, nvidia-) selects the GPU node
taint and labels.
required: [type, count]
properties:
type:
type: string
description: >-
GPU accelerator type (e.g. amd-mi355x,
nvidia-h100). Reported on the consuming
InferenceCluster's status.
minLength: 1
maxLength: 63
count:
type: integer
minimum: 1
maximum: 16
devices:
type: array
description: >-
Expand Down
130 changes: 129 additions & 1 deletion apis/inferenceclusters/definition.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,10 @@ spec:
x-kubernetes-validations:
- rule: "self.cluster.source != 'Nebius' || !has(self.nodePools) || self.nodePools.all(p, !has(p.fabric) || p.fabric.type != 'InfiniBand' || has(p.fabric.infiniband))"
message: fabric.infiniband is required when fabric.type is InfiniBand and cluster.source is Nebius.
- rule: "self.cluster.source != 'VultrBaremetal' || !has(self.nodePools) || self.nodePools.all(p, !has(p.maxNodeCount))"
message: node pools cannot autoscale when cluster.source is VultrBaremetal; bare metal pools are fixed size.
- rule: "self.cluster.source != 'VultrBaremetal' || !has(self.stack) || self.stack == 'Standard'"
message: only the Standard stack is supported when cluster.source is VultrBaremetal.
properties:
cluster:
type: object
Expand All @@ -49,12 +53,14 @@ spec:
message: spec.cluster.nebius is required when spec.cluster.source is Nebius.
- rule: "self.source != 'Vultr' || has(self.vultr)"
message: spec.cluster.vultr is required when spec.cluster.source is Vultr.
- rule: "self.source != 'VultrBaremetal' || has(self.vultrBaremetal)"
message: spec.cluster.vultrBaremetal is required when spec.cluster.source is VultrBaremetal.
properties:
source:
type: string
description: >-
Cluster provisioning method.
enum: [GKE, EKS, AKS, Nebius, Vultr, Existing]
enum: [GKE, EKS, AKS, Nebius, Vultr, VultrBaremetal, Existing]
existing:
type: object
description: >-
Expand Down Expand Up @@ -310,6 +316,128 @@ spec:
default: default
minLength: 1
maxLength: 253
vultrBaremetal:
type: object
description: >-
Vultr bare metal (k3s) cluster configuration.
Required when source is VultrBaremetal. Provisions
bare metal servers - one CPU-only management server
plus the GPU pools - and installs a k3s cluster
onto them over SSH. Bare metal has no autoscaling,
so pools are fixed size, and provisioning takes
tens of minutes.
required: [region, ssh]
properties:
region:
type: string
description: >-
Vultr region for all servers (e.g. ewr, ord).
Bare metal plan availability varies by region;
check with vultr-cli plans list --type vbm.
minLength: 1
maxLength: 32
management:
type: object
default: {}
description: >-
The CPU-only bare metal server that runs the
k3s server (the management plane).
properties:
plan:
type: string
default: vbm-6c-32gb-amd
description: >-
Vultr bare metal plan for the management
server. The default is a CPU-only plan
available in most regions that offer bare
metal.
minLength: 1
maxLength: 63
osId:
type: integer
default: 2284
description: >-
Vultr operating system ID installed on
every server. Defaults to Ubuntu 24.04 LTS
x64; list IDs with vultr-cli os list.
ssh:
type: object
description: >-
SSH key pair used to reach the servers. The
public key is registered with Vultr and
installed on every server; the private key
drives the k3s install over SSH.
required: [secretRef]
properties:
secretRef:
type: object
description: >-
Secret holding the SSH key pair. The Secret
must exist in the modelplane-system
namespace.
required: [name]
properties:
name:
type: string
minLength: 1
maxLength: 253
privateKeyKey:
type: string
default: ssh-privatekey
description: >-
Key within the Secret that holds the
private key.
minLength: 1
maxLength: 253
publicKeyKey:
type: string
default: ssh-publickey
description: >-
Key within the Secret that holds the
public key.
minLength: 1
maxLength: 253
username:
type: string
default: root
description: >-
SSH user the servers accept the key for.
Vultr installs keys for root by default.
minLength: 1
maxLength: 63
k3s:
type: object
default: {}
description: The k3s release installed on the servers.
properties:
channel:
type: string
default: v1.34
description: >-
k3s release channel. Defaults to the first
channel where Dynamic Resource Allocation
(how GPUs bind to pods) is generally
available.
minLength: 1
maxLength: 32
credentials:
type: object
description: >-
Vultr ProviderConfig or ClusterProviderConfig used to
authenticate to the Vultr API. Defaults to the
ClusterProviderConfig named default.
properties:
type:
type: string
default: ClusterProviderConfig
enum:
- ProviderConfig
- ClusterProviderConfig
name:
type: string
default: default
minLength: 1
maxLength: 253
stack:
type: string
default: Standard
Expand Down
13 changes: 13 additions & 0 deletions apis/k3sclusters/composition.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: k3sclusters.infrastructure.modelplane.ai
spec:
compositeTypeRef:
apiVersion: infrastructure.modelplane.ai/v1alpha1
kind: K3sCluster
mode: Pipeline
pipeline:
- functionRef:
name: modelplane-modelplanecompose-k3s-cluster
step: compose-k3s-cluster
Loading
Loading