ci(opencode-review): add 5 free Zen models across distinct providers as deep failover - #716
Conversation
…as deep failover Expand the anonymous OpenCode Zen free reviewer pool from 7 to 12 models by adding five zero-cost, tool_call-capable models from five distinct upstream providers: hy3-free (Tencent Hy), minimax-m3-free (MiniMax), glm-5-free (Zhipu GLM), kimi-k2.5-free (Moonshot Kimi), and qwen3.6-plus-free (Alibaba Qwen). Each is verified tool_call=true in the models.dev opencode registry and carries the reasoning high-effort options/variants structure the contract requires. They are appended after the active free models in OPENCODE_MODEL_CANDIDATES, so free-first ordering holds and they are only reached once the active free pool is exhausted -- the exact scenario (free-pool exhaustion blocking all PR reviews) this hardens against. Distinct providers mean distinct outage domains, so a bad day for one upstream no longer drains the whole free tier before the paid fallback. The per-run dynamic budget cap plus the fatal-provider-error fast-kill already bound worst-case failover latency, so extra candidates add resilience without multiplying bad-day cost. Also fix a pre-existing contract drift: the coverage-evidence assertion still pinned the download-artifact v7.0.0 SHA after dependabot bumped the workflow to v8.0.1 (#596); align the expected SHA with the workflow. Contract updated in lockstep (tests/test_opencode_agent_contract.py): the public-conditional candidate string, the candidates prefix, the full candidate_pairs ordering, and the set(free_models) equality. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AzUnTqFnQqhRbaopvDdag7
|
Warning Review limit reached
Next review available in: 48 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
Comment |
The contract previously verified tool_call=true for only three hand-picked free models (nemotron-3-ultra-free, deepseek-v4-flash-free, north-mini-code-free). The reviewer drives CodeGraph/web-search tooling, so any free-pool candidate that lacks tool_call cannot emit a structured review and would burn its failover slot before yielding -- exactly the silent regression a future pool edit could introduce. Extend the existing free-model loop to assert tool_call is True for every candidate, securing coverage of the invariant the whole free-first failover chain depends on. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AzUnTqFnQqhRbaopvDdag7
|
@coderabbitai review |
✅ Action performedReview finished.
|
OpenCode Review Overview
--> Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review-dispatch.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review-dispatch.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["Test: test_opencode_agent_contract.py"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test: test_opencode_agent_contract.py"]
R2 --> V2["targeted test run"]
|
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
Approval sufficiency: APPROVE - PR #716 ci(opencode-review): add 5 free Zen models across distinct providers as deep failover adds five opencode-free model definitions (hy3-free, minimax-m3-free, glm-5-free, kimi-k2.5-free, qwen3.6-plus-free) to .github/workflows/opencode-review-dispatch.yml and extends OPENCODE_MODEL_CANDIDATES and tests/test_opencode_agent_contract.py in lockstep; changed-file evidence inspected via the trusted current-head hunks of .github/workflows/opencode-review-dispatch.yml and tests/test_opencode_agent_contract.py.
Verification posture: trusted current-head bounded evidence only - direct source-tree reads were denied by the sandbox policy, so verification uses the authoritative Changed files, Focused changed hunks, CodeGraph, and Coverage execution evidence; repo-native contracts are test_commands python3 -m pytest tests and coverage_commands python3 -m coverage run -m pytest tests && python3 -m coverage report --show-missing --fail-under=100.
Linter/static: no changed Python lint surface beyond a test edit; the workflow models-block edit preserves the existing JSON-style shape (each entry closed with }, next key after a comma, outer braces unchanged), and Failed GitHub Check evidence records no completed failed checks at this head.
TDD/regression: the contract test was updated in the same commit as the workflow change - conditional_public_candidate, the candidates list, and the expected provider/model pairs were all extended with the identical five ids in identical order, so any future drift fails the assertions.
Coverage: Coverage execution evidence records Result: PASS with supported repository test suites passed for tests/test_opencode_agent_contract.py via the repository test command python3 -m pytest tests.
Docstring coverage: Coverage execution evidence records configured repository docstring gates passed or docstring coverage was advisory; this PR introduces no new callable/docstring surface.
DAG: flowchart LR; A[".github/workflows/opencode-review-dispatch.yml"] --> B["opencode-free models map (5 new ids)"]; B --> C["OPENCODE_MODEL_CANDIDATES public prefix"]; C --> D["opencode review dispatch runtime pool"]; C -.contract-assert.-> E["tests/test_opencode_agent_contract.py"]; E --> F["python3 -m pytest tests -> Coverage PASS"]; the diagram reflects the base-to-head changed flow.
PoC/execution: no OPENCODE_EXECUTION_RECEIPT lines exist in bounded evidence for browser/E2E tooling; the only trusted execution outcome is Coverage execution evidence Result: PASS; no PoC claim is made.
DDD/domain: config-domain change only (provider pool composition); no entity, aggregate, or domain-model surface is touched.
CDD/context: context boundary preserved - the five free candidates sit inside the is_private == 'false' public branch only, and the private-repo fallback tail ('' }}opencode/gpt-5.6-terra ...) is byte-identical to base.
Similar issues: changed-file history shows this PR follows the established pool-expansion pattern (NVIDIA NIM provider #672, free-pool tool_call contract f4efa05, free-tier timeout fix #683); no divergent convention is introduced.
Claim/concept check: the PR body table (hy3-free 190k/64k tool_call=true; five distinct upstream providers; free pool 7 -> 12) matches the workflow limit/tool_call/reasoning fields and the 7 pre-existing free ids plus 5 new ones.
Standards search: no external standards or numeric formulas apply; provider ctx/out limits are operator-configured hints bounded by the existing watchdog (OPENCODE_POOL_STEP_TIMEOUT_SECONDS 12000, OPENCODE_TOTAL_RETRY_BUDGET_SECONDS 11700); availability of the new free ids is an offline-unverifiable runtime fact with fallback absorption.
Compatibility/convention: no breaking API or schema change; the candidate string grammar and opencode-free/ multi-word kebab-case naming (hy3-free, minimax-m3-free, glm-5-free, kimi-k2.5-free, qwen3.6-plus-free) match the existing 7 free ids; no reserved-word, identifier-exposure, or enumeration surface is introduced.
Breaking-change/backcompat: existing candidates and the private-repo fallback tail are unchanged, so backcompat is preserved; mergeStateStatus blocked is a branch/check policy state, not a conflict (mergeable: true; not DIRTY or CONFLICTING).
Implementation completeness: all five new model entries carry complete name/tool_call/reasoning/options/variants/limit fields identical in shape to prior entries; no pass, NotImplementedError, or TODO-only placeholders are introduced.
Performance: pool additions stay within the bounded watchdog budget - one attempt per model, 5400s per-candidate cap, 12000s pool-step cap - so worst-case added latency is absorbed by the outer guard.
Developer experience: the DX surface is the review-dispatch workflow config plus its single-source-of-truth contract test; the test pattern keeps future pool edits self-checked without manual list maintenance.
User experience: the UX surface is the review-gate outcome for PR authors; the free-candidate expansion raises the chance of a model approval on public repos while the unchanged paid/private fallback preserves prior behavior.
Visual/DOM: non-web change; no DOM, ARIA, or console surface exists; the reviewed interaction surface is the workflow candidate-selection path and its test assertions.
Accessibility/i18n: no UI or locale surface changed; English workflow comments and PR body are consistent.
Supply-chain/license: no dependencies added or removed; no lockfile, package, SBOM, or audit surface changed.
Packaging: no package manifests changed; the only Python artifact is a test file under the existing tests/ layout; unpackaged_source_surfaces is empty.
Security/privacy: no secrets, auth, or tenant-isolation surface changed; free candidates run only on public repos (is_private == 'false' guard), preserving private-repo privacy; no identifier exposure or enumeration surface.
Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including .github/workflows/opencode-review-dispatch.yml, tests/test_opencode_agent_contract.py.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: CodeGraph/source-backed behavior map connects .github/workflows/opencode-review-dispatch.yml to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source claims require trusted bounded source evidence prepared outside the isolated model process; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: deterministic repair does not infer browser runtime execution; source-backed DOM/UI evidence and trusted workflow receipts were reviewed when present, and non-web surfaces used API/CLI/log/docs/workflow evidence instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.
Findings
No blocking findings.
Adversarial validation
{"status":"passed","probes":[{"path":".github/workflows/opencode-review-dispatch.yml","line":3764,"hypothesis":"The five new model definitions could diverge from the candidate list (wrong id, order, or limits), so the dispatch runtime or the contract test would fail at head.","attack_or_counterexample":"Candidate references an undefined or reordered opencode-free id (e.g., qwen3.6-plus-free defined in the models map but kimi-k2.5-free ordered before it in OPENCODE_MODEL_CANDIDATES).","evidence":"Trusted current-head focused diff at .github/workflows/opencode-review-dispatch.yml:3764 shows the five ids added to the opencode-free models map with the same field shape as prior entries, and the identical ordered ids appear in OPENCODE_MODEL_CANDIDATES at line 4310; trusted Coverage execution evidence records Result: PASS with supported repository test suites passed, so tests/test_opencode_agent_contract.py, which parses the workflow and asserts the same id set and order, passed at head f5bbf5ca65b0ec2df18ffc8c27a4f976ffad7b3a. source-line-sha256=b8f2c450b33fd2af327ed7b373bdf9d83042afec169fdee8b2c1eeb8b1f8687c","outcome":"falsified"},{"path":".github/workflows/opencode-review-dispatch.yml","line":4310,"hypothesis":"Adding five free candidates could leak opencode-free models into private repositories or break the conditional expression (malformed trailing space or quote) so the private path receives an empty or wrong pool.","attack_or_counterexample":"is_private == 'true' input: the expression should still evaluate to the unchanged private tail; alternatively a missing space before ' || '' }} could corrupt the split.","evidence":"Trusted current-head diff at .github/workflows/opencode-review-dispatch.yml:4310 shows the five opencode-free ids inserted inside the public-only conditional prefix ending ' ... opencode-free/qwen3.6-plus-free ' || '' }}' with the private-repo fallback tail ('' }}opencode/gpt-5.6-terra ...) byte-identical to base; the contract test asserts candidates_text.startswith(conditional_public_candidate) with the same prefix string, and trusted Coverage execution evidence records Result: PASS, falsifying the private-leak and malformed-expression hypotheses. source-line-sha256=fef27c5d233ce4c29557c0ad6d5133d36a8faaa4cbae51d7bb3470741a49bc2f","outcome":"falsified"},{"path":"tests/test_opencode_agent_contract.py","line":108,"hypothesis":"The contract test could drift from the workflow (one of the five ids omitted from conditional_public_candidate or from the expected pairs), so the startswith/removeprefix/expected-pairs assertions would fail or pass vacuously.","attack_or_counterexample":"Workflow adds opencode-free/hy3-free but the test string omits it, making candidates_text.startswith(conditional_public_candidate) fail and blocking the workflow change.","evidence":"Trusted current-head diff at tests/test_opencode_agent_contract.py:108 extends conditional_public_candidate, the candidates list, and the expected provider/model pairs with the identical five ids in the same order as the workflow line; trusted Coverage execution evidence records Result: PASS for the repository test command python3 -m pytest tests, so the startswith/removeprefix/split and expected-pairs assertions held and the workflow/test strings did not drift. source-line-sha256=af64cc17984f114ce090bad2a195b20fe40e37a9eeeac649963d1b1e6983ed16","outcome":"falsified"}],"residual_risk":"Provider-side availability, ctx/out limits, and per-day free-model caps for the five new ids are external facts not verifiable in this offline review; a rate-limited or unavailable free candidate consumes one 5400s attempt and falls through to the next candidate, with the pool watchdog (OPENCODE_POOL_STEP_TIMEOUT_SECONDS 12000, OPENCODE_TOTAL_RETRY_BUDGET_SECONDS 11700) and the unchanged paid/private fallback list bounding overall review-gate availability."}Evidence
- Result: APPROVE
- Reason: APPROVE - the five new opencode-free review candidates (hy3-free, minimax-m3-free, glm-5-free, kimi-k2.5-free, qwen3.6-plus-free) are consistent across the workflow models map, the OPENCODE_MODEL_CANDIDATES expression, and the contract test expectations; trusted Coverage execution evidence records Result: PASS, no failed GitHub checks, and no unresolved review threads exist at this head.
- Scope:
central OpenCode/Strix review-process - Changed files:
2 - Head SHA:
f5bbf5ca65b0ec2df18ffc8c27a4f976ffad7b3a - Workflow run: 30809627172
- Workflow attempt: 1
This approval path is limited to ContextualWisdomLab/.github central review-process self-repair.
2 participants
What
Expands the anonymous OpenCode Zen free reviewer pool from 7 → 12 models by adding five zero-cost,
tool_call-capable models from five distinct upstream providers:hy3-freetool_call=trueminimax-m3-freetool_call=trueglm-5-freetool_call=truekimi-k2.5-freetool_call=trueqwen3.6-plus-freetool_call=trueEach declares
tool_call = trueexplicitly in the models.devopencoderegistry (the strong case — not family-inherited), and each carries thereasoning+ high-effortoptions/variantsstructure the contract requires.Why
Free-pool exhaustion is the org-wide keystone risk: when every active free model 429s/fails,
opencode-reviewrecordsexhaustedand blocks all PRs. The new models are appended after the active free models inOPENCODE_MODEL_CANDIDATES, so:OPENCODE_DYNAMIC_RUN_TIMEOUT_CAP_SECONDS) plus the fatal-provider-error fast-kill (a delisted model is killed on its error, not waited out for the 1h cap) already bound bad-day cost, so extra candidates add resilience without multiplying it.The registry marks every other unconfigured
-freemodelstatus = "deprecated", so these five are added as deep-failover candidates (all still declaretool_call=trueand zero cost).Drive-by fix
A pre-existing contract drift:
test_opencode_agent_contract.pystill pinned thedownload-artifactv7.0.0 SHA in thecoverage-evidencejob assertion after dependabot bumped the workflow to v8.0.1 (#596). The expected SHA is aligned with the workflow (3e5f45b2…).Verification
pytest tests/test_opencode_agent_contract.py— 40 passed (candidate string, candidates prefix, fullcandidate_pairsordering, andset(free_models)equality all updated in lockstep)pytest tests/test_opencode_model_pool_runner.py tests/test_opencode_workflow_shell_syntax.py— 33 passedbash scripts/ci/test_strix_quick_gate.sh— PASSjq -nconfig block is valid JSON with 12 free models, alltool_call=true.🤖 Generated with Claude Code
https://claude.ai/code/session_01AzUnTqFnQqhRbaopvDdag7
Generated by Claude Code