core: add portable schema generation

[yihanzhu]

Tracks #155

Concern

Add the first inactive member of the portable core generation: shared schema limits, primitives, references, envelope rules, and the one role/capability/permission/evidence policy table.

Construction mode authorizes this Roadmap unit. It starts from exact base 38a26f5f046897c0455fef24874c5dbb40c20926 and produces exact head 87daa48d08e6d9ac21a356bd9c4ab0350da91e86.

Generation: g-14b7ad8ce54c3b8c585ff92063d71551ffc7394cc2294d0297bc7d2b8da2c386

Design input spec blob: c6511d96c1a5e6aed27ba2075b5add65c121f782

The registry entry records that spec blob and construction authorization commit 38a26f5f046897c0455fef24874c5dbb40c20926 in the existing parent_plan_merge_commit field.

Files

• Add the private, import-free generation module modules/schema.jq.
• Create the canonical append-only generation registry with its first unique entry.
• Add literal fixtures and the exact schema-owned migration ledger map.
• Add one pinned jq 1.6 test that the fixed CI runner discovers.
• Append every new restore-critical path to ci/required-files.txt.

No workflow, trust root, accepted artifact, public wrapper, public generation root, manager/template, install path, live caller, frozen plan, or PR #183 state changes.

Proof on exact head

• Pinned official jq 1.6 asset and release SHA-256 verified.
• Owned rules: 47/47.
• Direct valid, invalid, and boundary cases: 141/141.
• Private route probes: 13/13.
• Registry cases: 8/8.
• Private activation guard cases: 35/35.
• Raw/parsed numeric boundary cases: 8/8.
• Schema-owned review findings: 8/8 from the verified 34-row ledger.
• Schema-owned legacy assertions: 44/44 from the verified 279-row ledger.
• Full fixed runner: all 9 test scripts passed.
• ShellCheck 0.11.0 full repository sweep: clean.
• Structure check: clean.
• Rename gate: clean.
• git diff --check: clean.

The diff is larger than the soft line signal because it keeps the 52 frozen migration rows as readable data and provides direct proof for every owned rule and bound. Product code is 377 lines; the remaining lines are literal fixtures, ledger mapping, pinned-tool proof, and restore entries. It remains one implementation concern.

Numeric layer boundary

jq 1.6 discards an integral float's spelling when it parses 1.0, so schema.jq cannot honestly recover that lexeme. The raw canonical-byte gate rejects 1.0, 1e0, integer fields written as 1.0, and schema_version:1.0 because jq's canonical bytes differ. Parsed schema rejects non-integral numbers, out-of-domain values, and jq-preserved -0. The tests prove both layers without assigning lexical parsing to the schema module.

Inactivity

The generation is incomplete and private. There is no scripts/core-contract.sh, no generation contracts.jq, no non-test loader, no live profile or target, and no release, install, deployment, credential, or external action.

[cloudflare-workers-and-pages]

Deploying ystack with    Cloudflare Pages

Latest commit:	87daa48
Status:	✅  Deploy successful!
Preview URL:	https://0724a1af.fabrica-6yx.pages.dev
Branch Preview URL:	https://codex-portable-core-schema.fabrica-6yx.pages.dev

View logs

[yihanzhu]

Codex reviewer (cross-vendor, read-only)

Reviewed-head: 2c12633
Reviewed-base: 38a26f5
reviewer: operator-default @ high

Posted verbatim by codex-review.sh (codex exec review --json --base refs/codex-review/189-20707/base in an isolated temp worktree, sandbox forced read-only). Comments only — Codex never pushes, approves, or merges.

The schema accepts malformed identity and reference strings with trailing newlines, violating its fail-closed contract. The private activation guard can also miss plausibly named public callers.

Full review comments:

• [P1] Anchor token regexes to the true end of the string — /var/folders/rm/bkx4f4b91vqfn79vk02pcd140000gn/T/tmp.TXxOnXbabI/core/v1/generations/g-14b7ad8ce54c3b8c585ff92063d71551ffc7394cc2294d0297bc7d2b8da2c386/modules/schema.jq:143-145
  For JSON strings ending in \n, Oniguruma's $ matches before the final newline, so id_ok, sha256_ok, and the analogous OID, timestamp, media-type, and ReverseDNS checks accept malformed values such as a 40-hex commit ID followed by a newline. These values become identities and immutable refs downstream; use a true end-of-string anchor such as \z consistently.

• [P2] Stop excluding portable-core-prefixed callers — /var/folders/rm/bkx4f4b91vqfn79vk02pcd140000gn/T/tmp.TXxOnXbabI/scripts/test/portable-core-schema.test.sh:522-522
  When a live caller is named something like scripts/portable-core-loader.sh, this broad exclusion omits it from the scan, so the activation guard succeeds even if that script names the inactive generation. Scan all candidate files and exempt only the known private test paths.

[yihanzhu]

Codex reviewer (cross-vendor, read-only)

Reviewed-head: eb93e13
Reviewed-base: 38a26f5
reviewer: operator-default @ high

Posted verbatim by codex-review.sh (codex exec review --json --base refs/codex-review/189-40290/base in an isolated temp worktree, sandbox forced read-only). Comments only — Codex never pushes, approves, or merges.

The schema implementation is otherwise internally consistent, but its private-activation guard can miss both alternate jq imports and activation surfaces outside its fixed directory list. That undermines a required safety gate for subsequent construction changes.

Full review comments:

• [P1] Detect all imports of the private schema — /var/folders/rm/bkx4f4b91vqfn79vk02pcd140000gn/T/tmp.2UO2GGfwpt/scripts/test/portable-core-schema.test.sh:553-554
  When a non-test jq file uses another valid spelling such as import "schema" as s;, this grep records no hit and schema_loaders_ok remains true. The activation guard therefore allows a public caller of the incomplete generation while reporting success; detect the schema module independently of alias and whitespace rather than matching one literal statement.

• [P1] Scan every tracked activation surface — /var/folders/rm/bkx4f4b91vqfn79vk02pcd140000gn/T/tmp.2UO2GGfwpt/scripts/test/portable-core-schema.test.sh:534-537
  During a later private child, a caller or document outside this hand-picked list—for example under .claude/hooks, core, or a new docs directory—can name the generation and this guard still passes. Because the guard is meant to prevent any public or non-test activation before assembly, scan the tracked tree and explicitly allow only the registry, manifest, generation members, and private tests.

[yihanzhu]

Codex reviewer (cross-vendor, read-only)

Reviewed-head: ae877c4
Reviewed-base: 38a26f5
reviewer: operator-default @ high

Posted verbatim by codex-review.sh (codex exec review --json --base refs/codex-review/189-60286/base in an isolated temp worktree, sandbox forced read-only). Comments only — Codex never pushes, approves, or merges.

The schema accepts numeric encodings that the contract explicitly rejects, and its activation guard misses valid jq loading syntax. Both weaken fail-closed guarantees central to this change.

Full review comments:

• [P1] Reject integer-valued floating-point literals — /var/folders/rm/bkx4f4b91vqfn79vk02pcd140000gn/T/tmp.eXQaTYSt4L/core/v1/generations/g-14b7ad8ce54c3b8c585ff92063d71551ffc7394cc2294d0297bc7d2b8da2c386/modules/schema.jq:144-144
  When a document contains a canonical numeric literal such as schema_version: 1.0 or another integer field encoded as 1.0, jq preserves that spelling but reports it equal to floor, so both int_ok and schema_layer_ok accept it. -0 is accepted for the same reason. This violates the contract's requirement to reject floats and negative integers, so the integer check must distinguish canonical integer literals from numerically integral floats.

• [P2] Scan all legal jq module-loading forms — /var/folders/rm/bkx4f4b91vqfn79vk02pcd140000gn/T/tmp.eXQaTYSt4L/scripts/test/portable-core-schema.test.sh:545-545
  When a later tracked non-test caller loads this module with valid jq syntax such as include "schema"; or compact import"schema"as s;, this pattern does not add it to schema_import_hits, so the private-activation guard can pass despite an incomplete generation being callable. The accepted contract explicitly forbids include and non-test callers before assembly, so the scan must recognize every legal loading form rather than only spaced import ... as statements.

[yihanzhu]

Review response — exact head 350483f53e88d60f9030e6d7ceea14023bf88d2a

Both findings from review comment 5466080428 are addressed without widening the unit.

1. The tracked-tree scanner now recognizes an exact schema module load after either import or include. It accepts compact, spaced, tabbed, and multiline jq syntax and does not depend on the alias. Positive cases cover include "schema";, include"schema";, spaced/compact imports, arbitrary aliases, newlines, and metadata. Negative cases cover another module, schema-extra, and an identifier containing the word import.
2. Numeric rejection follows the real layer boundary. jq 1.6 normalizes parsed 1.0 to 1, so the schema module does not claim it can recover that lost spelling. The exact raw canonical-byte check rejects scalar 1.0, 1e0, an integer field written as 1.0, and schema_version:1.0. Parsed int_ok rejects fractions, negative/out-of-range values, and jq-preserved -0. A short invariant comment records that schema receives only values accepted by the raw gate.

Fresh proof on this head:

• owner rules 47/47;
• direct cases 108/108;
• raw/parsed numeric boundary cases 8/8;
• activation guard cases 25/25;
• full fixed runner 9/9 scripts;
• ShellCheck 0.11.0, structure, rename, and diff checks clean.

No accepted artifact, workflow, public wrapper/root, live caller, or frozen state changed.

[yihanzhu]

Codex reviewer (cross-vendor, read-only)

Reviewed-head: 350483f
Reviewed-base: 38a26f5
reviewer: operator-default @ high

Posted verbatim by codex-review.sh (codex exec review --json --base refs/codex-review/189-80090/base in an isolated temp worktree, sandbox forced read-only). Comments only — Codex never pushes, approves, or merges.

The schema accepts malformed newline-terminated primitive values despite its exact-format contract. Its import-free CI guard also has a concrete syntax bypass.

Full review comments:

• [P1] Anchor primitive regexes to the absolute string end — /var/folders/rm/bkx4f4b91vqfn79vk02pcd140000gn/T/tmp.rrpSWH9uKB/core/v1/generations/g-14b7ad8ce54c3b8c585ff92063d71551ffc7394cc2294d0297bc7d2b8da2c386/modules/schema.jq:143-143
  When a canonical document contains an identifier whose decoded value ends in \n, jq/Oniguruma lets $ match before that final newline, so id_ok returns true; the same defect affects the SHA, OID, media-type, DNS, and timestamp predicates. This violates the contract's exact formats and admits malformed refs, so use absolute \z anchors throughout and add trailing-newline regression cases.

• [P2] Detect compact imports in the import-free guard — /var/folders/rm/bkx4f4b91vqfn79vk02pcd140000gn/T/tmp.rrpSWH9uKB/scripts/test/portable-core-schema.test.sh:538-539
  When schema.jq gains valid compact jq syntax such as import"profile_graph"as profile_graph;, this grep misses it because it requires whitespace or ( after the keyword. The broader tracked-file scan only searches for imports of schema, so CI would still report this module as import-free despite its dependency table requiring no imports; broaden this check and cover compact forms.

[yihanzhu]

Review response — exact head 87daa48d08e6d9ac21a356bd9c4ab0350da91e86

Both findings from review comment 5466125146 are addressed within the schema unit.

1. Every exact product regex now uses Oniguruma absolute \A / \z anchors. This covers IDs, SHA-256, both Git OID widths, media type, every ReverseDNS label, timestamp matching/capture, and Git revision/object OIDs. Registry generation/spec/authorization proof uses the same absolute anchors. Named regressions inject a decoded trailing newline into each primitive and through every envelope/reference family.
2. The schema import-free guard now scans full jq directive syntax rather than a whitespace-dependent grep. It recognizes compact, spaced, and multiline import/include, arbitrary module strings and aliases, and metadata. Regressions include import"profile_graph"as profile_graph;, multiline imports, compact/multiline includes, metadata, and basic prose/string/comment non-directives. The full tracked-tree loader guard still decodes the quoted module string and blocks every non-allowlisted load of schema, including escaped module spelling.

Fresh exact-head proof:

• owned rules 47/47;
• direct cases 141/141;
• registry cases 8/8;
• activation guard cases 35/35;
• raw/parsed numeric boundary cases 8/8;
• full fixed runner 9/9 scripts;
• ShellCheck 0.11.0, structure, rename, and diff checks clean.

No workflow, accepted artifact, public root/wrapper, live caller, or frozen state changed.

[yihanzhu]

Codex reviewer (cross-vendor, read-only)

Reviewed-head: 87daa48
Reviewed-base: 38a26f5
reviewer: operator-default @ high

Posted verbatim by codex-review.sh (codex exec review --json --base refs/codex-review/189-247/base in an isolated temp worktree, sandbox forced read-only). Comments only — Codex never pushes, approves, or merges.

The schema implementation matches the accepted contract’s primitives, shared references, policy mappings, generation registry, and inactive-generation constraints. The added tests provide focused boundary, routing, registry, and activation-guard coverage without changing live behavior.