Skip to content

fix(curiocity): fail all-skipped runs as no runnable trials - #335

Open
atomicdjt wants to merge 8 commits into
griddynamics:mainfrom
atomicdjt:fix/326-no-runnable-trials
Open

fix(curiocity): fail all-skipped runs as no runnable trials#335
atomicdjt wants to merge 8 commits into
griddynamics:mainfrom
atomicdjt:fix/326-no-runnable-trials

Conversation

@atomicdjt

Copy link
Copy Markdown
Contributor

Fixes #326.
Summary
Treat a non-empty Curiocity run in which every resulting trial is skipped as having no runnable trials, returning ExitCode.CONFIG_ERROR (2) instead of success.
This preserves the existing semantics for individual skipped trials: a skipped trial remains non-fatal when at least one other trial is runnable.
Changes
return exit code 2 when every selected trial resolves to skipped
preserve existing mixed runnable + skipped behavior
add unit coverage for both cases
add an end-to-end regression covering the missing/mistyped-agent path from #326
Verification
Verified on commit b80f055:
CI Curiocity ✅
type validation ✅
package build ✅
full Curiocity test suite ✅
CodeQL Advanced ✅
The implementation intentionally reuses the existing CONFIG_ERROR / exit-code-2 contract for “no runnable trials” rather than changing per-trial skipped semantics or introducing a new exit code.

@github-actions github-actions Bot added the bug Something isn't working label Aug 30, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Rosetta Triage Review

Summary: Fixes #326 — a Curiocity suite where every selected trial resolves to skipped (e.g. all agent ids mistyped) now returns ExitCode.CONFIG_ERROR (2) from gatekeeper() instead of 0, matching the documented contract in docs/architecture.md §13. Mixed runnable+skipped suites keep prior behavior (a lone skipped trial stays non-fatal). Also adds a repository == 'griddynamics/rosetta' guard to repo-implement.yml/repo-plan.yml so those scheduled/issue-triggered automation jobs don't fire on forks.

Findings:

  • Root cause and fix match the issue's own diagnosis: run.ts:244 guarded on matrix.length === 0 (never true when agent ids are merely unresolvable), not on the built trial specs; the new check in gatekeeper() (trials.length > 0 && trials.every(status === 'skipped')) closes that gap at the aggregation layer, which is shared by both run and report (D8), so a re-gated old run also gets the corrected exit code.
  • Test coverage is solid: a unit case for all-skipped, a unit case confirming one skipped + one runnable trial still exits 0, and an integration test reproducing the exact [ROSETTA] curiocity exits 0 when a run executes zero trials, so CI can pass on nothing #326 scenario (mistyped agent → 1 skipped trial → exit 2).
  • The fork-guard changes to repo-implement.yml/repo-plan.yml are unrelated to the stated PR scope (curiocity gatekeeper fix) but are low-risk, defensive additions consistent with existing repo-scoping patterns elsewhere (this repo's automation is board-driven and org-scoped already).

Caveats:

  • Scope mixing: the PR bundles an unrelated CI/workflow change (fork-guard on two automation workflows) with the curiocity bug fix. Consider splitting in the future for cleaner history/review, though not blocking here given the small, low-risk diff.

Automated triage by Rosetta agent

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ROSETTA] curiocity exits 0 when a run executes zero trials, so CI can pass on nothing

1 participant