test(parity): retry parity sides on transient docker build flakes#11
Merged
Conversation
The runtime parity lane's docker builds flake on contended CI runners (BuildKit "failed to solve", a registry hiccup, or a container-setup failure surfaced as `Command failed: docker build`). A flake on one side while the other succeeds shows up as a false TS-vs-Go divergence and fails the job under PARITY_STRICT=true, where an inconclusive skip is red too. This turned unrelated dependabot PRs (#9, #10) and a main daily run red on up.dotfiles-local* cases. Wrap each side's CLI invocation in a bounded retry (2 attempts) gated on isRetryableFailure, a predicate deliberately broader than isInfraError. Retrying is safe: a deterministic product failure reproduces on the next attempt and still goes red. The narrow isInfraError/classifyParitySide oracle-usability path is unchanged, so no real divergence is masked as a skip. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR hardens the runtime parity test harness against transient Docker/BuildKit flakes by retrying each parity “side” invocation when failures match known infra signatures, reducing false TS-vs-Go divergences in task parity:runtime under PARITY_STRICT=true.
Changes:
- Wrap parity CLI execution in a bounded retry loop (
runWithInfraRetry) keyed off a newisRetryableFailureclassifier (broader thanisInfraError, but used only for retry decisions). - Introduce
parityInfraRetriesto cap total attempts per side. - Add unit tests covering retry classification and retry-loop behavior without requiring Docker.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| internal/cli/parity_matrix_test.go | Adds bounded infra retry around parity side execution and introduces isRetryableFailure. |
| internal/cli/parity_retry_test.go | Adds focused unit tests for retryability classification and retry-loop behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Manuel Alejandro de Brito Fontes <aledbf@gmail.com>
Comment on lines
+1258
to
+1273
| func isRetryableFailure(stdout, stderr string) bool { | ||
| if isInfraError(stdout, stderr) { | ||
| return true | ||
| } | ||
| combined := strings.ToLower(stdout + stderr) | ||
| retryablePatterns := []string{ | ||
| "an error occurred setting up the container", | ||
| "command failed: docker build", | ||
| } | ||
| for _, p := range retryablePatterns { | ||
| if strings.Contains(combined, p) { | ||
| return true | ||
| } | ||
| } | ||
| return false | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The runtime parity lane (
task parity:runtime,PARITY_STRICT=true) turned red on unrelated changes — dependabot PRs #9 and #10, plus the main daily run on 2026-07-14 — all onup.dotfiles-local*cases. None were product regressions:docker build("An error occurred setting up the container. Command failed: docker build ...") →TS=1 Go=0→ false divergence.parity_matrix_test.go:351.up.dotfiles-local-successsignature, no PR involved.These are transient docker/BuildKit failures on contended CI runners. Under
PARITY_STRICT=truethere is no classification that makes them green — aninconclusiveskip also fails the strict gate — so the only correct fix is to make the affected side actually succeed via a bounded retry.Change
runParityCLInow wraps each side's CLI invocation inrunWithInfraRetry(up toparityInfraRetries = 2attempts). The original body isrunParityCLIOnce.isRetryableFailure, deliberately broader thanisInfraError(adds the container-setup /docker buildwrapper). Used only to decide a retry — the narrowisInfraError/classifyParitySideoracle-usability path is unchanged, so no real divergence is masked as a skip.Tests (
parity_retry_test.go, no docker required)TestIsRetryableFailure: the exact PR chore(deps): bump the go-dependencies group across 1 directory with 2 updates #9/chore(deps): bump the actions group across 1 directory with 2 updates #10 signatures classify retryable; genuine contract errors (config-not-found, invalid flag, empty output) do not.TestRunWithInfraRetry: transient-then-success is absorbed; a stable contract failure is never retried; a persistent transient failure still surfaces red after the cap; a cancelled context stops retries.Validation
go test ./internal/cli -run 'TestIsRetryableFailure|TestRunWithInfraRetry'→ 12 passedgo test ./internal/cli -run 'TestClassifyParitySide|TestParityReport|TestStrictParityError|...'→ 8 passed (existing helpers unaffected)task lint(go vet) → passNot run:
task parity:runtime(needs Docker + containerd image store, ~60 min); the flake can't be reproduced deterministically locally, so coverage is unit-level over the retry decision and loop where the regression lives.🤖 Generated with Claude Code