Skip to content

Reject empty result arrays so an empty eval run cannot pass CI #3

Description

@royalpinto007

Problem

POST /api/runs checks that results is an array, but accepts an empty array. The run is stored with total: 0, and the response reports shouldFail: false. A broken CI producer can therefore submit no evaluation cases and still get a passing gate.

Relevant code: app/api/runs/route.ts.

Expected behavior

An empty results array should return HTTP 400 with a clear validation error and should not create a suite or run.

Acceptance criteria

  • Reject results: [] before any database write.
  • Keep non-empty payload behavior unchanged.
  • Add a regression test for the empty-array case.
  • Document the requirement in the informational GET /api/runs response if needed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggood first issueGood for newcomers

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions