Skip to content

fix(ci): eliminate 141 redundant prover jobs without losing exercised tests - #136

Closed
hyperpolymath wants to merge 2 commits into
mainfrom
fix/backend-matrix-honest-wiremock-20261029
Closed

hyperpolymath wants to merge 2 commits into
mainfrom
fix/backend-matrix-honest-wiremock-20261029

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

Measured CI problem

VCL-UT's Backend Matrix expanded Echidna's 141 prover slugs into 141 Rust jobs (max-parallel: 3; GitHub listed 142 jobs including the prep job). The PR waits behind at least 47 waves of almost identical jobs. This is distinct from Gitleaks, which usually completes in seconds once it starts.

The per-prover smoke was not real per-prover verification: it passed ECHIDNA_PROVER into each job but neither src/interface/echidna-client/src/ nor tests/ reads the environment variable. Every job runs the same cargo test --manifest-path src/interface/echidna-client/Cargo.toml --tests suite against wiremock endpoints. These tests establish transport semantics for OpenTheory/Dedukti and HTTP errors, not behaviour of 141 distinct prover engines. The prior UI advertised per-backend evidence that did not exist.

Fix

  • Keep a CI step fetching Echidna's authoritative .machine_readable/provers.a2ml, now validating nonempty, correctly shaped, uniquely named slugs and consistency with its declared count.
  • Replace the 141 identical Rust jobs with one identical wiremock transport suite, with the same Rust toolchain, echidna-core sibling checkout, pinned actions, and required cargo test command.
  • Add an explicit fail-closed source guard: if ECHIDNA_PROVER begins appearing in the client source or tests, the single-suite lane fails and demands genuine prover-specific assertions / restoration of the matrix before a green claim.
  • Keep all PR/push/manual triggers and the Backend Matrix workflow identity; no test suite is disabled, no CI bypass, and no source-level proof claim is made.

Verification

  • Repository source search: selector is referenced only in the old workflow, not client code/tests.
  • actionlint .github/workflows/backend-matrix.yml: PASS.
  • git diff --check: PASS.
  • Commit 0ee0c7c signed and verified (G).

The actual client cargo test and live GitHub Actions run must confirm the revised workflow. This change intentionally does not purport to test distinct prover implementations; adding real per-prover semantics requires a separate substantive test design.

@coderabbitai

coderabbitai Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 48 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: bff04f3e-8df8-426d-8a1b-585f863d12b4
Commits

Reviewing files that changed from the base of the PR and between 034a057 and ce72e63.

Files selected for processing: 1
  • .github/workflows/backend-matrix.yml
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-29T22:39:59.186787Z 0ee0c7c PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@sonarqubecloud

Copy link
Copy Markdown

@hyperpolymath

Copy link
Copy Markdown
Owner Author

Superseded by merged #133 (commit 591e6f3). Both the prover-independent transport gate and PR-scoped cancellation commits were cherry-picked, and the replacement two-job Backend Matrix passed on the combined branch. Verified the new workflow on main; closing this separate PR to avoid duplicate changes, not representing it as independently merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant