Skip to content

fix(vitest-cloud): retry transient post-execute status/log queries - #660

Merged
danshapiro merged 1 commit into
mainfrom
fix/vitest-cloud-status-retry
Aug 18, 2026
Merged

fix(vitest-cloud): retry transient post-execute status/log queries#660
danshapiro merged 1 commit into
mainfrom
fix/vitest-cloud-status-retry

Conversation

@danshapiro

Copy link
Copy Markdown
Owner

What

After gcloud run jobs execute --wait completes, the wrapper queries execution status to decide success. Right at that moment the executions describe call can transiently fail — observed live 2026-08-18 on a run where all 4 shards finished green: the wrapper exited 1 purely because that single query errored.

Fix

  • Retry each status query up to 5×3s before declaring failure (still fails closed when results are truthfully unavailable).
  • Retry an empty post-run log read once, so the printed per-shard summary isn't silently lost to the same transient window.

Test plan

  • scripts/test/cloud-vitest-wrapper.test.sh: new regression checks — a fake gcloud fails the first two describe calls, run must still exit 0; full suite passes locally.
  • Live-validated against the real backend: full cloud vitest suite at 667b655 (4 shards, ~4700 tests, 0 failures) with the image rebuilt per-HEAD.

Observed 2026-08-18 on a fully-green 4-shard run (execution ended with
succeededCount=4): the wrapper exited 1 because the gcloud executions
describe for succeededCount errored once right after execute --wait
returned. Retry each status query up to 5x3s and retry an empty log read
once, so a transient CLI flake cannot fail an otherwise-green cloud run.
Regression-covered in cloud-vitest-wrapper.test.sh (fake gcloud fails the
first two describe calls; run must still succeed).
@danshapiro
danshapiro merged commit 5556c05 into main Aug 18, 2026
3 checks passed
pull Bot pushed a commit to HinchK/freshell that referenced this pull request Aug 18, 2026
…xecute output

gcloud prints 'Execution [NAME] has successfully completed.' with literal
brackets and ANSI SGR codes around the name; the wrappers parsed with
grep -oP 'Execution \K[^ ]+', capturing the bracket (and escapes). Every
downstream describe/logs call then addressed a nonexistent execution:
- vitest-cloud.sh: status queries all failed, so a fully green 4-shard run
  exited 1 (observed live 2026-08-18, executions freshell-vitest-xzrwg and
  -ftrdv; the 5x retry from danshapiro#660 could not help a deterministic miss).
- e2e-cloud.sh: '|| echo 0' masked it into succeeded=0 — the wrapper would
  report success even when the status query never reached the real execution.

Fix: strip ANSI escapes and accept the bracketed form
('Execution \[?\K[A-Za-z0-9][A-Za-z0-9-]*'). Regression-tested in
scripts/test/cloud-exec-id-parse.test.sh for both wrappers (exit code,
truthful succeeded count, clean id in downstream describe calls), verified
RED before the fix and green after; cloud-vitest-wrapper.test.sh and
cloud-vitest-entrypoint.test.sh remain green.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant