Skip to content

chore(test): one generator registry behind build:check, one Io capture fake, one ROOT, conventions stated - #200

Merged
Vivswan merged 13 commits into
mainfrom
chore/test-helpers-and-generator-registry
Sep 13, 2026
Merged

Vivswan merged 13 commits into
mainfrom
chore/test-helpers-and-generator-registry

Conversation

@Vivswan

@Vivswan Vivswan commented Sep 13, 2026 •

Copy link
Copy Markdown
Owner

Before / After

# before: build:check spelled the generated files by hand; src/upstream-gaps/index.ts was not in it
"build:check": "bun run build:schema && git diff --exit-code lib/settings.schema.json && ... README.md COVERAGE.md action.yml docs/... | grep ."

# after: one table, every generator, every registered path
"build:check": "bun .github/scripts/generated.ts"
$ bun run build:check
build:check: 12 generated files match their generators

What changed

One registry of generated output drives build:check, and the test suite shares one Io capture fake, one temp-dir shape, and one ROOT instead of per-file copies.

  • build:check now gates src/upstream-gaps/index.ts with the schema, the docs regions, and action.yml; on drift, git diff --stat lists the stale files and the check exits 1.
  • A page carrying a BEGIN GENERATED marker must be in the table, and vice versa; the test scans the tree through the shared marker grammar.
  • CONTRIBUTING.md states the conventions: line caps, the markdown one-line rule, colocated section tests, the file-header rule, the shared test helpers; "the one committed generated artifact" now names the table.

Proof

  • Tests and gate: bun run check green (3426 tests); the drift runner has a negative control (a clean clone passes, staged stale bytes in a region file and a whole file exit 1 naming both); bun run build:lib && bun .github/scripts/package-smoke.ts ok.
Technical details

Per item

item what count
Generator registry .github/scripts/generated.ts (75 lines) derives GENERATED_OUTPUTS (path, generator, kind) from PAGE_REGIONS, GENERATED_REGIONS, and the gaps INDEX_PATH; the runner runs every generator, then git diff --exit-code --stat and git ls-files --others over the registered paths (git's own output names the files; a failing ls-files fails the check). test/scripts/generated.test.ts (117) is one tree-scan pin plus two controls (marker grammar, clone drift runner) 12 paths, 4 generators
Io capture fake test/io/capture.ts (57) captureIo(onMask?): annotations, logs, outputs, masks, and one ordered event log (annotate <level>: ..., log: ..., summary: <first line>, output <name>=<value>, mask: <value>) 7 local fakes and 8 inline const io: Io = {...} literals deleted; 10 importers
Temp-dir shape test("...", () => withTempDir("prefix-", async (dir) => {...})) from test/temp-dir.ts (20); helpers reading a shared dir take it as a parameter 12 files lose their afterEach/afterAll hooks
ROOT test/root.ts (5) 29 declarations and 10 inline join(import.meta.dir, "..", "..", ...) spellings replaced; #199's workflow-loader.ts copy swept too
Conventions CONTRIBUTING.md Code conventions and Tests sections; AGENTS.md's generated-artifacts bullet names the table

Line accounting (62 files, +2380 / -2276)

kind lines residual
tests +2279 / -2270 the helpers (capture.ts 57, temp-dir.ts 20, root.ts 5) and generated.test.ts (117) are paid for by the deleted fakes, literals, hooks, and ROOT copies; snapshot.test.ts +31 and run.test.ts +20 are one withTempDir line per temp-dir test (20 and 14) plus helpers returning the path they compute
scripts +85 / -4 generated.ts replaces the 700-character build:check one-liner; generated-regions.ts +6 exports hasGeneratedRegion()
docs +16 / -2 CONTRIBUTING.md, AGENTS.md

Conventions stated: biome's 100 for code; the fleet's check-file-size 256 cap over source, test, workflow, and shell lines (markdown prose has no width cap); 10-line comment blocks; markdown one source line per item; section tests in src/sections/<key>/, the rest under test/ mirroring src/; every temp dir removed on every exit path (withTempDir() or a try/finally; the release-pipeline fixture in afterAll and the e2e bundle on process exit are the suite-scoped exceptions).

Small fixes on the way: toBe(null) -> toBeNull() and the one new URL(...) sibling read -> join(import.meta.dir, ...) in test/e2e/openapi/validate.test.ts.

Rebases: onto #201 (033a9c6), #204 (f1f938e), #199 (798fc42); each a pure replay except the hunks main's edits overlapped (deliver.test.ts and redact.test.ts event lists, the temp-dir files, the ROOT declarations), redone against main's text.

Codex rubber-duck review, 12 rounds: the branch (2), the gate-fix delta (3), each rebase's resolutions (3, all clean), the two trims (4; the last found the runner discarding git ls-files' exit status, fixed). Every finding fixed in its round.

Recorded, not built

  • Header comments on the 26 import-first files under src/ and .github/scripts/ (the rule is stated; the sweep is its own change).
  • test/action/run.test.ts's module-level testIo literal (40+ assertions read its mixed-format array).
  • Converting the ~14 hand-rolled mkdtempSync + try/finally sites to withTempDir().
  • The drift-runner control proves the changed paths, not byte-exact restoration; the dropped shape checks (package.json's build:check line, per-path single writer, inputs.md row count) are not implied by the pin.
  • No biome rule for toHaveLength(0) vs toEqual([]).

Copilot AI balanced review requested due to automatic review settings September 13, 2026 07:39
@github-actions

github-actions Bot commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor

File size check

0 over a hard cap (fails), 7 warning(s).

File Size Tier Cap
.github/scripts/release-pipeline.ts 1755 lines warn 1600
.github/scripts/release-pipeline.ts:339 168 chars warn 150
.github/scripts/release-pipeline.ts:1 26 comment lines (header) warn 25
.github/scripts/release-pipeline.ts:1572 13 comment lines warn 10
.github/workflows/post-green.yml:139 11 comment lines warn 10
test/action/run.test.ts:759 151 chars warn 150
test/docs/guides.test.ts:468 155 chars warn 150

Split the file, wrap the line, shorten or exempt the comment, or list the path in .file-size-allow.local with a # reason.

4 managed file(s) skipped; repo-platform owns them.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The generator gate and mechanical test-helper migrations are internally consistent and retain the existing assertions.

Pull request overview

Centralizes generated-output validation and shared test infrastructure while documenting repository conventions.

Changes:

  • Adds a generator registry powering build:check, including upstream-gap output.
  • Consolidates test I/O capture, temporary directories, and repository-root resolution.
  • Updates conventions and migrates affected tests.
File summaries
File Description
.github/scripts/generated.ts Adds the generated-output registry and drift gate.
.github/scripts/gen-docs.ts Exports the coverage output path.
.github/scripts/gen-gaps-index.ts Exports the gaps index path.
package.json Routes build:check through the registry.
AGENTS.md Updates generated-artifact guidance.
CONTRIBUTING.md Documents code and test conventions.
test/temp-dir.ts Adds shared temporary-directory helpers.
test/root.ts Adds the shared repository root.
test/io/capture.ts Adds the shared I/O capture fake.
test/scripts/generated.test.ts Verifies registry completeness and wiring.
test/scripts/gen-docs.test.ts Delegates page-set coverage to the registry test.
test/scripts/gen-action-docs.test.ts Uses shared ROOT.
test/scripts/graduate-upstream-gaps.test.ts Uses shared ROOT.
test/scripts/release-pipeline.test.ts Uses shared ROOT.
test/scripts/check-compat-markers.test.ts Adopts shared temp directories and ROOT.
test/scripts/changed-sections.test.ts Adopts shared temp directories and ROOT.
test/sections/graphql-queries.test.ts Uses shared ROOT.
test/sections/docs-registry.test.ts Uses shared ROOT.
test/report/delivery.test.ts Uses shared I/O capture.
test/published-schema.test.ts Uses shared ROOT.
test/library/public-surface.test.ts Uses shared ROOT.
test/io/mask-pair.test.ts Uses shared I/O capture.
test/flows/snapshot.test.ts Moves temp-directory lifetime into test bodies.
test/flows/redact.test.ts Uses the unified ordered I/O event format.
test/flows/multi.test.ts Uses shared I/O and temp-directory helpers.
test/flows/merge.test.ts Uses per-test temporary directories.
test/flows/layers.test.ts Uses per-test temporary directories.
test/flows/execute.test.ts Uses shared ROOT and temp directories.
test/flows/deliver.test.ts Uses the unified I/O event format.
test/engine/snapshot.test.ts Uses shared I/O capture.
test/engine/secrets.test.ts Uses shared I/O and silent fakes.
test/engine/orchestrate.test.ts Uses shared I/O capture and mask hooks.
test/e2e/schema.ts Uses shared ROOT for section scenarios.
test/e2e/runner.ts Uses shared ROOT.
test/e2e/openapi/validate.test.ts Uses the idiomatic null matcher.
test/e2e/foundation.test.ts Uses shared ROOT for source scanning.
test/docs/version-pins.ts Uses shared ROOT.
test/docs/semantics.test.ts Uses shared ROOT.
test/docs/readme.test.ts Uses shared ROOT.
test/docs/post-green-workflow.test.ts Uses shared ROOT.
test/docs/permissions.test.ts Uses shared ROOT.
test/docs/npm-publish-workflows.test.ts Uses shared ROOT.
test/docs/library-examples.test.ts Uses shared ROOT.
test/docs/issue-templates.test.ts Uses shared ROOT.
test/docs/guides.test.ts Uses shared ROOT.
test/docs/e2e-nightly-workflow.test.ts Uses shared ROOT.
test/docs/diagrams.test.ts Uses shared ROOT.
test/docs/coverage.test.ts Uses shared ROOT.
test/docs/ci-workflow.test.ts Uses shared ROOT.
test/docs/checks-workflow.test.ts Uses shared ROOT.
test/docs/checks-sections-gate.test.ts Uses shared ROOT.
test/docs/biome-schema.test.ts Uses shared ROOT.
test/cli/program.test.ts Adopts shared temp directories and ROOT.
test/cli/io.test.ts Moves temporary working directories into test bodies.
test/cli/init.test.ts Adopts shared temporary-directory helpers.
test/cli/execution.test.ts Scopes action and CLI fixtures with withTempDir.
test/architecture/architecture.test.ts Uses shared ROOT.
test/action/run.test.ts Migrates merge and snapshot fixtures to shared helpers.
test/action/io.test.ts Uses shared temp directories and ROOT.
test/action/action-yml.test.ts Uses shared ROOT.
src/sections/branches/branches.test.ts Uses shared I/O capture.
src/sections/actions_secrets/actions_secrets.test.ts Uses shared I/O capture and mask timing.
Review details
  • Files reviewed: 62/62 changed files
  • Comments generated: 0
  • Review effort level: Balanced (auto)

Note

Copilot is running an experiment and ran this review at Balanced.


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@Vivswan
Vivswan marked this pull request as ready for review September 13, 2026 07:44
@Vivswan Vivswan added the merge-when-green Owner approved: merge once every gate is green label Sep 13, 2026
Copilot AI review requested due to automatic review settings September 13, 2026 08:16

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The centralized helpers preserve existing assertions, and the new registry is comprehensively guarded against missing outputs and drift.

Review details
  • Files reviewed: 63/63 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced (auto)

Note

Copilot is running an experiment and ran this review at Balanced.

Copilot AI review requested due to automatic review settings September 13, 2026 08:40
@Vivswan
Vivswan force-pushed the chore/test-helpers-and-generator-registry branch from 5d7ae08 to 07b918c Compare September 13, 2026 08:40

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The centralized tooling preserves existing behavior and includes targeted drift and regression coverage.

Review details
  • Files reviewed: 63/63 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced (auto)

Note

Copilot is running an experiment and ran this review at Balanced.

Copilot AI review requested due to automatic review settings September 13, 2026 08:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The registry runner and mechanical test-helper migrations are coherent, covered by focused controls, and introduce no unresolved defects.

Review details
  • Files reviewed: 63/63 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced (auto)

Note

Copilot is running an experiment and ran this review at Balanced.

Before: package.json's build:check spelled ten generated paths by hand and
never checked src/upstream-gaps/index.ts.
After: .github/scripts/generated.ts derives GENERATED_OUTPUTS (path, generator,
kind) from the generators' own registries; build:check runs every generator
in the table and diffs every registered path, the gaps index included.

- test/scripts/generated.test.ts pins the table against the tree: every
  tracked .md/.yml file carrying a BEGIN GENERATED marker is registered and
  vice versa, the two whole-file outputs are tracked with one writer each.
- The hand-typed page list in gen-docs.test.ts goes; the tree scan is the pin.
Before: seven local captureIo/recordingIo/eventIo copies and six inline
`const io: Io = {...}` literals, each recording a different subset of the
port in its own format.
After: captureIo() records every channel into its own list and into one
ordered event log; the copies and literals are deleted, every assertion
kept (the event strings move to the one format: "annotate <level>: ...",
"log: ...", "mask: ...").
Before: twelve test files created temp directories in beforeEach or a
scratch list and removed them in afterEach/afterAll hooks.
After: test/temp-dir.ts exports withTempDir() (try/finally, removed on
every exit path) and tempDirTest() (a test handed its own dir); the hooks
are gone, and the helpers that read a shared `dir` take it as a parameter.
CONTRIBUTING.md states the rule in one line.
Before: CONTRIBUTING.md called lib/settings.schema.json "the one committed
generated artifact" and stated no line caps, test placement, or file-header
rule.
After: the committed generated output is the table in
.github/scripts/generated.ts; a Code conventions section states the line
caps (100 for code, 256 elsewhere, 10-line comment blocks), the
one-line-per-paragraph markdown rule, the colocated-test rule, and the
file-header convention; the Tests section names the shared fakes.
Before: 26 verbatim `const ROOT = join(import.meta.dir, "..", "..")` copies, a
REPO_ROOT, a depth-one copy, and eight inline `join(import.meta.dir, "..", "..",
...)` spellings across test/.
After: test/root.ts exports ROOT once; every repository-relative path in a test
resolves from it. A file beside its test still resolves from import.meta.dir.
On the way: toBe(null) -> toBeNull() in test/e2e/openapi/validate.test.ts.
…mp-dir rule as it holds

Before: "comments, YAML, and prose stay under the fleet's 256-character cap"; "no afterEach or afterAll hook cleans up" a temp dir.
After: the 256-character width cap is check-file-size's, over source, test, workflow, and shell lines; markdown prose has no width cap. Every temp dir is removed on every exit path, through the shared helpers or a try/finally of its own; the one suite-scoped fixture cleans in afterAll.
…r, and control the drift runner

Before: the tree scan matched a line-anchored BEGIN marker; the grammar also admits an inline HTML marker (inputs.md's outputs-list), so an inline-only page could leave the table unnoticed. The runner's negative control was manual.
After: generated-regions.ts exports hasGeneratedRegion(), the scan reads it, and unit controls pin the inline, non-comment, and block-scalar cases. A test clones HEAD to a temp dir: the clean clone passes the runner, and a staged stale byte in docs/reference/sections.md and in src/upstream-gaps/index.ts makes it exit 1 naming both.
…ot HEAD's

Before: the temp clone ran HEAD's .github/scripts/generated.ts, so an uncommitted edit that broke the runner still passed the control.
After: the working tree's .github/scripts is copied over the clone before each run; the postconditions exclude that copy and the node_modules link.
Before: tests +140 net; tempTest("long title", async (dir) => {...}) wrapped to three header lines because biome applies test-call formatting only to test/it; generated.test.ts carried shape checks the pin implies.
After: tests +0 net. One temp-dir shape, test("title", () => withTempDir(prefix, async (dir) => {...})); tempDirTest() deleted. generated.test.ts is the tree-scan pin and the two controls; generated.ts is the table and the runner; capture.ts builds its lists once.
…ries capture

Before: the runner re-listed drifted paths in its own modified:/untracked: format; captureIo kept a summaries list no test read.
After: git diff --exit-code --stat prints the drifted paths, ls-files prints an untracked one, and one error line follows; a failing ls-files fails the check. captureIo's summary channel lands in events only.
Copilot AI review requested due to automatic review settings September 13, 2026 09:48
@Vivswan
Vivswan force-pushed the chore/test-helpers-and-generator-registry branch from 8f064a5 to 29c96a0 Compare September 13, 2026 09:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The registry, drift controls, helper migrations, and documentation are internally consistent and adequately tested.

Review details
  • Files reviewed: 62/62 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced (auto)

Note

Copilot is running an experiment and ran this review at Balanced.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-when-green Owner approved: merge once every gate is green

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants