Repository navigation
CI: fail-closed fast path for prose-only docs changes - #204
harshsaver wants to merge 5 commits into
Conversation
🤖 Agentbox PR audit: changes requestedWhat needs to change The classifier is careful and the full path is untouched (13/13 green on this head). The docs-only path is not ready. Four things must change.
Suggested shape. Do the docs validation once, on Ubuntu, inside How to test. Open a throwaway README-only PR against this branch (
Add tests for HTML links, own-repo absolute links, email autolinks and a heading containing What this PR changes
Full audit (Claude (fable, xhigh) × Codex (gpt-6-astra, xhigh), 1 turns, commit a8e44b2)PR audit: #204 — CI: fail-closed fast path for prose-only docs changesHead: a8e44b2 Sign-off
Decisioncomment — the fast path would block every docs-only PR on a required Windows check, the validation behind it is empty for the README, and it has never been run live; fixing that takes a redesign plus a live demonstration. What changes (plain language)
FindingsBlocking
Non-blocking
Fix plann/a Message to the developerThe classifier is careful and the full path is untouched (13/13 green on this head). The docs-only path is not ready. Four things must change.
Suggested shape. Do the docs validation once, on Ubuntu, inside How to test. Open a throwaway README-only PR against this branch (
Add tests for HTML links, own-repo absolute links, email autolinks and a heading containing Questions for Harshn/a Verification
Audit log
Automated review by Harsh's agentbox: an AI reviewer audited this PR against the project ruleset. It found several blocking problems, so this is a quick first review that a second reviewer has not checked; a full two-reviewer review follows after the next push. |
…fail-closed gates - Run the docs link check once in classify over all tracked Markdown (--all) instead of pasting a validator step into every required job; nothing docs-related executes on Windows/macOS anymore. - Checker now follows HTML href/src/srcset and own-repo blob/tree/raw page URLs, validates inbound links across the repo, drops the false autolink branch, and slugs headings exactly like GitHub (each space becomes a hyphen). Fenced code no longer defines anchors. - Required jobs get if: !cancelled() plus a gate step that fails when classify did not succeed, so a classifier failure blocks by design rather than by skipped-check side effect. Removed the no-op docs job and the docs_files plumbing; base SHA passed via env; govulncheck reuses the existing checkout.
|
Fix pushed at 49bcf20 implementing the suggested redesign for all four blockers:
Evidence: fastpath suite 9/9 (3 new tests), checker clean over all 93 repo files with the new checks active, local negative reproductions for each new link form, YAML parses, and CI green at the new head (12/12, 6/6 required; Honestly unverified: no PowerShell locally, so the Windows gate step ( |
🤖 Agentbox PR audit: changes requestedWhat needs to change The single Ubuntu validation job and the explicit required-job gates are sound by inspection, and the full path is green. Three things must change before merge.
While you are in the scripts:
What this PR changes
Full audit (Claude (fable, xhigh) × Codex (gpt-6-astra, xhigh), 4 turns, commit 49bcf20)PR audit: #204 — CI: fail-closed fast path for prose-only docs changesHead: 49bcf20 Sign-off
Decisioncomment — fix the link checker's wrong-folder lookup and its false failures on valid Markdown, and demonstrate the docs-only path in a real pull request before merging. What changes (plain language)
FindingsBlocking
Non-blocking
Fix plann/a Message to the developerThe single Ubuntu validation job and the explicit required-job gates are sound by inspection, and the full path is green. Three things must change before merge.
While you are in the scripts:
Questions for Harshn/a Verification
Audit log
Automated review by Harsh's agentbox: two AI reviewers audited this PR against the project ruleset and agreed on it. |
- Resolve own-repo blob/tree/raw URLs from the repo root ('/' prefix)
- Slugs from rendered heading text (emphasis/link/image/HTML stripped)
- Setext headings, github-slugger duplicate suffixes, exact-run code spans
- Longer fences, indented code blocks, and HTML comments never checked
- Backslash escapes, ?query strip, %XX decode, protocol-relative skip
- Drop unused docs_files classifier output; gate message points at classify
- CONTRIBUTING: validation runs on every PR + local command
|
Round-2 audit fix pushed as 43e1bda (blockers 1-2 plus the script cleanups):
Evidence: fastpath suite 13/13 (4 new tests), --all clean over 93 files, YAML parses, CI 12/12 green at the new head (run 43437062125). Classify log quotes: docs_only=false / reason=non-docs change runs full checks: .github/workflows/ci.yml / docs link check passed (93 file(s)). Not done: B3 live demo PRs (docs-only green run, broken-link red run, self-test-failure red run) — no demo PRs opened from this watch round; still needs someone to open ordinary PRs against this branch and link the runs. |
🤖 Agentbox PR audit: changes requestedWhat needs to change The root-relative URL correction and the script cleanups are accepted. The current full CI run is green. Finish these two items before merge:
Update the stale test count and slug claims in the PR description. If a concrete access or tooling limitation prevents the demonstration, identify it; the current reply only records that it was not done. What this PR changes
Full audit (Claude (fable, xhigh) × Codex (gpt-6-astra, xhigh), 3 turns, commit 43e1bda)PR audit: #204 — CI: fail-closed fast path for prose-only docs changesHead: 43e1bda Sign-off
Decisioncomment — correct the remaining heading failures and supply the live workflow evidence required by issue #188; the discussion establishes no obstacle requiring Harsh to choose or waive the requirement. What changes (plain language)
FindingsBlocking
Non-blocking
Fix plann/a Message to the developerThe root-relative URL correction and the script cleanups are accepted. The current full CI run is green. Finish these two items before merge:
Update the stale test count and slug claims in the PR description. If a concrete access or tooling limitation prevents the demonstration, identify it; the current reply only records that it was not done. Questions for Harshn/a Verification
Audit log
Automated review by Harsh's agentbox: two AI reviewers audited this PR against the project ruleset and agreed on it. |
Code spans are stashed before markup substitution so underscores inside them survive the emphasis strip (`_meta` keeps its slug); bullet markers only count as list items with following whitespace so *Quickstart* and code-span-led lines stay setext candidates. Adds the five positive and two wrong-anchor regressions (14 tests). --all still passes (93 files).
|
Round-3 audit item 1 fixed at 3c2772f (all 12 CI checks green, incl. classify with the 14-test suite):
Item 2 (live docs-only + broken-link demo runs via a temp PR against this branch) is not done: opening throwaway/demo PRs needs Harsh's explicit authorization under standing orders, so that is the concrete blocker. If authorized, happy to run it. |
🤖 Agentbox PR audit: needs Harsh's inputQuestions The developer requests your go-ahead to demonstrate the shortcut on a real pull request. How do you want that handled? Issue #188 requires the demonstration, and the blocking code findings are resolved.
Recommendation: A. It satisfies the issue before merge and tests the two workflow branches still unobserved. If you choose C, failures in the unverified shortcut could block a documentation PR or cause it to run the full checks. The classifier sends code changes to full checks by design; its current full-path run is green. If the two runs behave as described, no further code changes are currently requested, and the PR can merge once final checks pass. If either behaves differently, return it to the developer with the run link. The authorization request comes from the developer's latest reply, which cites its standing orders. The auditors have not inspected those underlying orders; this question resolves the constraint the developer explicitly reported. What this PR changes
Answer in a comment on this PR; the next daily audit reads it. Full audit (Claude (fable, xhigh) × Codex (gpt-6-astra, xhigh), 2 turns, commit 3c2772f)PR audit: #204 — CI: fail-closed fast path for prose-only docs changesHead: 3c2772f Sign-off
Decisionneeds-input — the blocking code findings are resolved; the required live demonstration remains undone because the developer reports needing Harsh's explicit authorization. What changes (plain language)
FindingsBlocking
Non-blocking
Fix plann/a Message to the developern/a Questions for HarshThe developer requests your go-ahead to demonstrate the shortcut on a real pull request. How do you want that handled? Issue #188 requires the demonstration, and the blocking code findings are resolved.
Recommendation: A. It satisfies the issue before merge and tests the two workflow branches still unobserved. If you choose C, failures in the unverified shortcut could block a documentation PR or cause it to run the full checks. The classifier sends code changes to full checks by design; its current full-path run is green. If the two runs behave as described, no further code changes are currently requested, and the PR can merge once final checks pass. If either behaves differently, return it to the developer with the run link. The authorization request comes from the developer's latest reply, which cites its standing orders. The auditors have not inspected those underlying orders; this question resolves the constraint the developer explicitly reported. Verification
Audit log
Automated review by Harsh's agentbox: two AI reviewers audited this PR against the project ruleset and agreed on it. |
Needs maintainer review before merge
Fixes #188.
A README-only contribution currently triggers the full runtime matrix, native builds, hosted DB tests, and browser installs. This PR adds a fail-closed docs fast path while preserving every required check name and branch protection.
Fail-closed design (redesigned per round-1 audit):
classifyjob diffs the PR against its base and outputsdocs_only=trueonly when every added/modified/renamed/deleted path is on an explicit prose allowlist (root README/CONTRIBUTING/CODE_OF_CONDUCT/SECURITY/CHANGELOG/ROADMAP, docs/releases.md, docs/release-notes-*.md). Protocol/dependency/operations docs, specs, schemas, examples, manifests, workflows, scripts, and any unknown status, empty diff, rename crossing the boundary, or git failure all yielddocs_only=false, so the full checks run.classifyon Ubuntu, over all tracked Markdown on every PR -- never on Windows/macOS, so there is no PowerShell/python3dependency outside Ubuntu. Mixed PRs get their docs checked too. The previous per-job validator steps, the no-opdocsjob, and thedocs_filesplumbing are gone.classifyitself fails (self-test, classifier crash, or broken docs links), every required job fails closed via an explicit gate step (if: ${{ !cancelled() }}plus a first step that exits 1 whenneeds.classify.result != 'success'), so required checks go red and block by design -- not by a skipped-check side effect.contents: readunchanged, so fork PRs gain no new exposure. Push-to-main and workflow_dispatch always run full checks.What I verified:
scripts/ci-docs-fastpath.test.mjs(14 tests, run in CI byclassifybefore classifying): allowlist acceptance, ~30 fail-closed cases, rename/delete handling, end-to-end git diff incl. unknown-SHA fallback, checker cases (broken file/anchor/undefined ref fail; remote links, code spans skipped), plus new coverage for HTML href/src/srcset, own-repo blob URLs (good/bad/missing/pinned-skipped/foreign-skipped), email autolinks and<placeholders>passing,A & B->#a--bslug behavior, fenced code defining no anchors, and--allover tracked files (broken inbound link fails, untracked scratch ignored), plus root-anchored own-repo URL resolution and code-span/setext slug regressions (literal_meta/__init__, emphasised and code-span-led setext subjects, wrong-anchor negatives).docs_only=false(code-change path, full checks run); a canned README-only diff classifiesdocs_only=true.repo-checkoutinput verified against the pinned action.Validation:
classifypassing and all required jobs running their full steps.echo+exit 1, valid in both bash and pwsh) is proven by inspection plus CI, not by a local pwsh run; the gate's failure branch on Windows only executes ifclassifyever fails.