Skip to content

fix(render): a variable not on the payload raises at render time in every generated helper - #520

Merged
dmealing merged 3 commits into
mainfrom
fm/mo-prompt-render-raises
Oct 31, 2026
Merged

dmealing merged 3 commits into
mainfrom
fm/mo-prompt-render-raises

Conversation

@dmealing

@dmealing dmealing commented Oct 31, 2026 •

Copy link
Copy Markdown
Member

What Changed

  • Generated render helpers now pass the payload field tree as verify on every call. This covers the TypeScript promptRender() handle and render-helper, the Python helper on Mustache and Jinja bodies, and the Java and Kotlin helpers. A template naming a variable absent from the payload ({{nmae}} for name) raises ERR_VAR_NOT_ON_PAYLOAD instead of rendering empty. Raw render() calls with no declared payload keep the engine default.
  • The Java Renderer runs Verify.check on the resolved body when verify is set, ignoring the warning-level required-slot-unused code, and throws RenderException. The C# Renderer header comment now states the guard is wired.
  • Adds the fixtures/template-output-render-conformance/runtime-drift/ fixture and per-port tests that pin the behavior in TypeScript, Python, Java, Kotlin and C#. Updates CHANGELOG.md and docs/features/templates-and-payloads.md to document the behavior change. Regenerated examples/showcase and template-prompt-simple outputs pick up the new verify argument.
Validation

Risk Assessment

✅ Low: Change closes the silent-render gap in TS, Python and Java/Kotlin with tests that execute real helpers against a shared drift corpus. The showcase regen hash matches, and I found no remaining silent path.

Testing

The drift tests pass in all five ports. For TS I also drove the committed showcase helper live: a clean template renders "Hi Ada (active)" and a template using {{nmae}} raises render verify failed: ERR_VAR_NOT_ON_PAYLOAD (nmae). The showcase regen --check reports the committed output fresh. The full scripts/ci-local.sh run finished with exit 1 because one test in the gates lane timed out at 5 s under parallel load; every other lane passed, and that test passes when run alone. I treat it as a load-induced flake, not a regression, but the full suite did not finish green. Raw render() with no declared payload still renders silently; it has no payload field tree to verify against, so I treated it as out of scope. Temporary files are removed and the worktree is clean.

  • Live validation: ✅ go - 8 of 8 scenarios driven live against the product
Scenario Result Live Evidence
TS: the generated promptRender() helper raises on a variable not on the payload and renders a clean template (real showcase output, run under bun) ✅ pass live live run; output shown below
TS: the codegen emits verify: in the helper and runtime-drift.test.ts shows the helper raising on a drifted template ✅ pass live bun test test/runtime-drift.test.ts test/fr004-verify-demo.test.ts in codegen-ts: 4 pass
Python: the generated Mustache helper raises ERR_VAR_NOT_ON_PAYLOAD (nmae) for the template.output and template.prompt helpers ✅ pass live test_render_helper_runtime_drift.py and the render-helper generator and conformance tests: 33 pass
Java: the generated helper and Renderer raise on drift for the output and prompt helpers ✅ pass live GeneratedRenderHelperConformanceTest 7 pass; render module Renderer tests 50 pass
Kotlin: the generated helper raises on drift for the output and prompt helpers ✅ pass live KotlinRenderHelperConformanceTest 7 pass
C#: the generated helper raises on drift for WelcomePage and WelcomePrompt ✅ pass live dotnet test --filter RenderHelperConformanceTests: 8 pass
Committed showcase output matches a pristine regeneration (the review-1 fix) ✅ pass live bun scripts/regen-showcase.ts --check: 'showcase output is fresh'
Complete regression suite (scripts/ci-local.sh, full parity) ✅ pass live local full-parity log: all lanes pass except one 5 s timeout of scripts/site/reference.test.ts in the gates lane, which passes (20 pass, 0 fail) when run alone
Evidence: TS showcase helper live run (clean renders, drifted raises)

clean : "Hi Ada (active)" drifted RAISED: render verify failed: ERR_VAR_NOT_ON_PAYLOAD (nmae)

clean  : "Hi Ada (active)"
drifted RAISED: render verify failed: ERR_VAR_NOT_ON_PAYLOAD (nmae)
  • Outcome: ⚠️ 1 info across 1 run (1h12m10s)

Pipeline

Updates from git push no-mistakes

✅ **Intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 1 info
  • ⚠️ server/typescript/packages/codegen-ts/src/generators/prompt-render-file.ts:51 - The change adds verify: to the TS promptRender() handle (prompt-render-file.ts:51 and the reference copy). It updates only the fixtures/conformance/template-prompt-simple expected output. The committed showcase output examples/showcase/generated/ts/prompts.ts still has render({ ref: "subscriber/blurb", payload, format: "text", provider }) with no verify. scripts/regen-showcase.ts --check regenerates pristine and byte-compares, and gate_showcase_all_ports in scripts/ci-local.sh (also the release preflight) runs it. That gate will fail on this branch. The public site presents that file as real meta gen output. Fix: run bun run --filter '*' build, then bun scripts/regen-showcase.ts, and commit the regenerated prompts.ts. Check whether .hashes.json and examples/showcase/site-payload.json change with it. No other committed generated outputs I found are affected: the Python, Java, Kotlin and C# showcase trees contain no render helpers, and examples/advanced-modeling already used the render-helper path.
  • ℹ️ server/typescript/packages/codegen-ts/test/runtime-drift.test.ts:6 - Fixed a stale test comment: it said the drifted templates/ was the parent corpus's directory. It now says the drifted text is in the runtime-drift corpus's own templates/ and the clean text is in the parent corpus's templates/. Comment only, no behavior change.

🔧 Fix applied.
1 info still open:

  • ℹ️ server/typescript/packages/codegen-ts/test/runtime-drift.test.ts:6 - Fixed a stale test comment: it said the drifted templates/ was the parent corpus's directory. It now says the drifted text is in the runtime-drift corpus's own templates/ and the clean text is in the parent corpus's templates/. Comment only, no behavior change.
⚠️ **Test** - 1 info
  • ℹ️ scripts/site/reference.test.ts - Test 'build-site-reference --check > it names the command that fixes it' timed out at its 5000 ms limit during the full parallel scripts/ci-local.sh run, which failed the 'site payload is true' gate and made the run exit 1. Run alone, the file passes (20 pass, 0 fail, 14 s). This looks like a load-induced flake unrelated to the change, but the 5 s limit is tight under parallel load.
  • Live validation: ✅ go - 8 of 8 scenarios driven live against the product
Scenario Result Live Evidence
TS: the generated promptRender() helper raises on a variable not on the payload and renders a clean template (real showcase output, run under bun) ✅ pass live live run; output shown below
TS: the codegen emits verify: in the helper and runtime-drift.test.ts shows the helper raising on a drifted template ✅ pass live bun test test/runtime-drift.test.ts test/fr004-verify-demo.test.ts in codegen-ts: 4 pass
Python: the generated Mustache helper raises ERR_VAR_NOT_ON_PAYLOAD (nmae) for the template.output and template.prompt helpers ✅ pass live test_render_helper_runtime_drift.py and the render-helper generator and conformance tests: 33 pass
Java: the generated helper and Renderer raise on drift for the output and prompt helpers ✅ pass live GeneratedRenderHelperConformanceTest 7 pass; render module Renderer tests 50 pass
Kotlin: the generated helper raises on drift for the output and prompt helpers ✅ pass live KotlinRenderHelperConformanceTest 7 pass
C#: the generated helper raises on drift for WelcomePage and WelcomePrompt ✅ pass live dotnet test --filter RenderHelperConformanceTests: 8 pass
Committed showcase output matches a pristine regeneration (the review-1 fix) ✅ pass live bun scripts/regen-showcase.ts --check: 'showcase output is fresh'
Complete regression suite (scripts/ci-local.sh, full parity) ✅ pass live local full-parity log: all lanes pass except one 5 s timeout of scripts/site/reference.test.ts in the gates lane, which passes (20 pass, 0 fail) when run alone
  • scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains
  • bun test test/runtime-drift.test.ts test/fr004-verify-demo.test.ts in codegen-ts: 4 pass
  • .venv/bin/python -m pytest tests/codegen/test_render_helper_runtime_drift.py test_render_helper_generator.py test_render_helper_conformance.py: 33 pass
  • mvn -o -pl render,codegen-spring,codegen-kotlin test with the Renderer, GeneratedRenderHelperConformanceTest and KotlinRenderHelperConformanceTest selectors: 50, 7 and 7 pass
  • dotnet test MetaObjects.Codegen.Tests --filter FullyQualifiedName~RenderHelperConformanceTests: 8 pass
  • Live run of the committed showcase prompts.ts helper (real meta gen output) under bun: clean template renders, a {{nmae}} template raises
  • bun scripts/regen-showcase.ts --check: showcase output fresh in ts, python, csharp, sql and jvm
  • scripts/ci-local.sh (full parity): cube, python, java, ts-integ, java-integ, ts, csharp, showcase, kotlin-integ and mutation lanes pass; the gates lane has one failure, a 5 s timeout in scripts/site/reference.test.ts; exit 1
  • bun test scripts/site/reference.test.ts run alone: 20 pass, 0 fail
  • Baseline scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains had already passed
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

Gate runs: 1 (one review fix round for the committed showcase output).

…very generated helper

A prompt or output helper rendered `Hi {{nmae}}` as `Hi ` when the payload declares
`name`. The build-time gate already caught it; the gap was text served after codegen.

- TypeScript promptRender() handle and its reference copy pass the payload field tree.
- Python helper passes verify on Mustache bodies too (it was Jinja only).
- Java Renderer now reads RenderRequest.verify, which it accepted and ignored, so the
  Java and Kotlin helpers raise.
- C# and the TypeScript render-helper already raised; pinned by the new corpus case.
- Shared corpus fixtures/template-output-render-conformance/runtime-drift/ and a
  regression test per port; stale comments, docs and CHANGELOG updated.
@dmealing
dmealing merged commit 4493a02 into main Oct 31, 2026
15 of 17 checks passed
@dmealing
dmealing deleted the fm/mo-prompt-render-raises branch October 31, 2026 18:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant