Skip to content

spec.replicas is bounded (minimum 1, maximum 10) but neither bound is documented #438

Description

@nicknikolakakis

What happened?

docs/content/models/model-deployment.md (lines 222-226) says:

spec.replicas is the only scaling axis. Each replica is a complete, fixed-shape serving instance, so scaling adds or removes whole instances across the fleet. Because the deployment exposes the Kubernetes scale subresource, kubectl scale and KEDA work without anything extra. There's no in-cluster pod autoscaling.

apis/modeldeployments/definition.yaml (lines 41-44) bounds the field at both ends:

replicas:
  type: integer
  minimum: 1
  maximum: 10

So scaling to zero is rejected:

$ kubectl -n ml-team scale modeldeployment mock-demo-v2 --replicas=0
The ModelDeployment "mock-demo-v2" is invalid: spec.replicas:
  Invalid value: 0: spec.replicas in body should be greater than or equal to 1

Neither bound appears in the docs. "Sizing a deployment" and "Scaling" in model-deployment.md both describe spec.replicas without mentioning a range, and getting-started/scale-the-model.md doesn't either.

Two separate surprises, and the ceiling is the bigger one:

  • maximum: 10 caps a deployment at ten replicas fleet-wide. For a project whose premise is spreading a model across many GPU clusters, that ceiling seems worth stating in the docs, or raising if it's just a placeholder guard.
  • minimum: 1 means scale-to-zero isn't reachable. Scale-to-zero is one of the main reasons people put KEDA in front of an expensive GPU workload, so naming KEDA and the scale subresource in the same paragraph sets an expectation the API doesn't meet.

I don't know whether either bound is deliberate. If they are, a sentence next to the KEDA mention would save people the discovery. If the minimum isn't, dropping it to 0 would make a parked-but-defined deployment expressible.

For context on why zero is useful: I wanted to withdraw one deployment's endpoints while leaving the object in place, to observe how a weighted ModelService behaves when an entry matches no healthy endpoints. I had to taint the hosting cluster instead, which is a heavier instrument and also moves replicas.

Note this is about the ModelDeployment replica count, not GPU node pools. #345 covers pools not scaling to zero on clouds using the vanilla cluster-autoscaler, which is a separate layer.

How can we reproduce it?

  1. Apply any ModelDeployment.
  2. kubectl -n <ns> scale modeldeployment <name> --replicas=0
  3. The API server rejects it with the message above.

What environment did it happen in?

Modelplane version: v0.3.1 (xpkg.upbound.io/modelplane/modelplane:v0.3.1)
Crossplane: 2.4.0
Kubernetes: kind, kindest/node:v1.34.0
Inference clusters: source: Existing (BYO, registered by kubeconfig)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions