What happened?
docs/content/models/model-deployment.md (lines 222-226) says:
spec.replicas is the only scaling axis. Each replica is a complete, fixed-shape serving instance, so scaling adds or removes whole instances across the fleet. Because the deployment exposes the Kubernetes scale subresource, kubectl scale and KEDA work without anything extra. There's no in-cluster pod autoscaling.
apis/modeldeployments/definition.yaml (lines 41-44) bounds the field at both ends:
replicas:
type: integer
minimum: 1
maximum: 10
So scaling to zero is rejected:
$ kubectl -n ml-team scale modeldeployment mock-demo-v2 --replicas=0
The ModelDeployment "mock-demo-v2" is invalid: spec.replicas:
Invalid value: 0: spec.replicas in body should be greater than or equal to 1
Neither bound appears in the docs. "Sizing a deployment" and "Scaling" in model-deployment.md both describe spec.replicas without mentioning a range, and getting-started/scale-the-model.md doesn't either.
Two separate surprises, and the ceiling is the bigger one:
maximum: 10 caps a deployment at ten replicas fleet-wide. For a project whose premise is spreading a model across many GPU clusters, that ceiling seems worth stating in the docs, or raising if it's just a placeholder guard.
minimum: 1 means scale-to-zero isn't reachable. Scale-to-zero is one of the main reasons people put KEDA in front of an expensive GPU workload, so naming KEDA and the scale subresource in the same paragraph sets an expectation the API doesn't meet.
I don't know whether either bound is deliberate. If they are, a sentence next to the KEDA mention would save people the discovery. If the minimum isn't, dropping it to 0 would make a parked-but-defined deployment expressible.
For context on why zero is useful: I wanted to withdraw one deployment's endpoints while leaving the object in place, to observe how a weighted ModelService behaves when an entry matches no healthy endpoints. I had to taint the hosting cluster instead, which is a heavier instrument and also moves replicas.
Note this is about the ModelDeployment replica count, not GPU node pools. #345 covers pools not scaling to zero on clouds using the vanilla cluster-autoscaler, which is a separate layer.
How can we reproduce it?
- Apply any
ModelDeployment.
kubectl -n <ns> scale modeldeployment <name> --replicas=0
- The API server rejects it with the message above.
What environment did it happen in?
Modelplane version: v0.3.1 (xpkg.upbound.io/modelplane/modelplane:v0.3.1)
Crossplane: 2.4.0
Kubernetes: kind, kindest/node:v1.34.0
Inference clusters: source: Existing (BYO, registered by kubeconfig)
What happened?
docs/content/models/model-deployment.md(lines 222-226) says:apis/modeldeployments/definition.yaml(lines 41-44) bounds the field at both ends:So scaling to zero is rejected:
Neither bound appears in the docs. "Sizing a deployment" and "Scaling" in
model-deployment.mdboth describespec.replicaswithout mentioning a range, andgetting-started/scale-the-model.mddoesn't either.Two separate surprises, and the ceiling is the bigger one:
maximum: 10caps a deployment at ten replicas fleet-wide. For a project whose premise is spreading a model across many GPU clusters, that ceiling seems worth stating in the docs, or raising if it's just a placeholder guard.minimum: 1means scale-to-zero isn't reachable. Scale-to-zero is one of the main reasons people put KEDA in front of an expensive GPU workload, so naming KEDA and the scale subresource in the same paragraph sets an expectation the API doesn't meet.I don't know whether either bound is deliberate. If they are, a sentence next to the KEDA mention would save people the discovery. If the minimum isn't, dropping it to 0 would make a parked-but-defined deployment expressible.
For context on why zero is useful: I wanted to withdraw one deployment's endpoints while leaving the object in place, to observe how a weighted
ModelServicebehaves when an entry matches no healthy endpoints. I had to taint the hosting cluster instead, which is a heavier instrument and also moves replicas.Note this is about the
ModelDeploymentreplica count, not GPU node pools. #345 covers pools not scaling to zero on clouds using the vanilla cluster-autoscaler, which is a separate layer.How can we reproduce it?
ModelDeployment.kubectl -n <ns> scale modeldeployment <name> --replicas=0What environment did it happen in?
Modelplane version: v0.3.1 (
xpkg.upbound.io/modelplane/modelplane:v0.3.1)Crossplane: 2.4.0
Kubernetes: kind,
kindest/node:v1.34.0Inference clusters:
source: Existing(BYO, registered by kubeconfig)