Add upstream-shaped scoped introspection plugin

[Zetazzz]

Summary

This is an upstream-shaped, opt-in alternative to #1719. It copies the exact Crystal release sources pinned by CNC (pg-introspection@1.0.1, graphile-build-pg@5.1.3) and keeps the upstream package boundaries while adding a scoped query and replacement introspection plugin.

Unset/false uses untouched upstream stock introspection. GRAPHILE_SCOPED_INTROSPECTION=true dynamically installs ScopedIntrospectionPreset; GRAPHILE_SCOPED_INTROSPECTION_JIT independently controls the introspection session JIT setting and defaults to false.

Architecture and isolation

• pg-introspection: parameterized recursive schema/dependency-closure query, all/dependency-closure catalog type policy, extension capability retention.
• graphile-build-pg: complete CNC-owned PgScopedIntrospectionPlugin, upstream lifecycle/watch/entity behavior, namespace and missing-type fail-closed guards, pinned upstream contract sentinel.
• CNC wiring: boolean config from env/server/service; stock startup does not load the scoped package; ConstructivePreset remains unchanged.
• Benchmark arms differ only by upstream PgIntrospectionPlugin/stock query versus ScopedIntrospectionPreset/scoped query. They do not load ConstructivePreset, retirement/disposers, governor/cache limits, routing, or admission plugins.

Correctness coverage

The real PostgreSQL dependency-closure fixture now covers cross-schema enum/domain/composite/range/multirange types, function signatures, ordinary and pg_trgm indexes, extension metadata, a cross-schema foreign key, and inheritance direction. It proves the required dependency parent is retained while a reverse parent-to-child relationship does not pull an unrelated child into the closure. No production query-algorithm change was needed.

Catalog benchmark

The scoped-only benchmark reuses the generic fresh-process runner/protocol/report. It adds:

• fixed 65-table root surface plus dependency and unrelated noise schemas;
• scalable noise tables, indexes, sequences, types, and functions;
• explicit identical JIT off/on settings for stock and scoped;
• public gather-hook parsed entity counts;
• a separate query-only worker for SQL time, UTF-8 payload bytes, entity counts, and bind count;
• paired p50/p95/min/max and median percent-change analysis;
• raw reports in the gitignored packages/perf-harness/artifacts/ directory.

Environment: macOS arm64, Node v22.22.0, PostgreSQL 18.4 in a dedicated local container. PostgreSQL catalog cache is labeled shared-server-not-reset; this is not a pristine-cache cold-start claim. Every arm has 10 fresh-process samples. All 240 worker PIDs were unique.

JIT off (primary)

Negative percentages mean scoped reduced the metric.

Size	pg_class	Build ms stock → scoped p50/p95	Paired Δ	Heap MiB stock → scoped p50/p95	RSS MiB stock → scoped p50/p95	Peak RSS MiB stock → scoped p50/p95
small	825	322/333 → 329/342	+1.9%	49.1/49.1 → 46.7/46.7	152.9/153.4 → 150.2/150.7	154.2/154.8 → 152.6/153.1
medium	5,631	626/669 → 341/347	-45.8%	110.0/110.0 → 46.7/46.7	251.9/259.8 → 150.0/150.5	253.2/261.2 → 152.3/152.9
large	65,037	4,682/5,066 → 398/430	-91.5%	839.6/839.6 → 46.7/46.7	1,027.2/1,210.2 → 149.0/150.5	1,094.5/1,383.8 → 151.5/152.9

Size	Query ms stock → scoped p50/p95	Paired Δ	JSON bytes stock → scoped	Payload Δ	Parsed classes stock → scoped
small	56/60 → 60/66	+6.0%	1,372,051 → 1,060,547	-22.7%	276 → 270
medium	270/277 → 64/71	-76.5%	13,684,742 → 1,060,616	-92.2%	4,948 → 270
large	3,116/3,214 → 111/147	-96.5%	163,949,368 → 1,060,547	-99.4%	64,220 → 270

JIT on (diagnostic)

Size	Build ms stock → scoped p50/p95	Paired Δ	Query ms stock → scoped p50/p95	Paired Δ
small	326/337 → 690/699	+109.9%	49/57 → 1,713/1,725	+3,318.4%
medium	618/639 → 703/719	+13.3%	264/292 → 1,726/1,746	+552.0%
large	4,509/4,607 → 763/811	-83.2%	2,984/3,268 → 1,769/1,845	-40.9%

This diagnostic shows why JIT must be controlled: compilation cost dominates the recursive scoped query at small/medium scale. The primary comparison uses identical JIT-off sessions, matching scoped introspection's default.

Schema/runtime results

Both JIT matrices produced one stock/scoped-equivalent schema hash per size, and every runtime query passed:

• small: d31e8b1c2d9cab3d1e67dcbc734e5addd237637028089221cf5f014b9ede4aa5
• medium: 8c4e2e03008c98295978712461b799d23957ae89071326a16421ac4a7d0b1996
• large: 41c6ed8feb087766680ab1dc4d833e5082dbf69322b5fb2a118515e7c7ff9db0

Scoped reports retained exactly the root/dependency namespaces and excluded noise. Observed build/query session JIT values matched the requested value in every run.

Raw report SHA-256

• small: build-off d5624f57a72262993edf3f9b6168f7440d9ed80d1df7d00c5f550f571a810d42; query-off 2601f884973b0f7fd0f70efa412e1aca60159134b9358e35c4de9427dca59eb5; build-on 9d4d50aeca03d494cacd8d227ee78621f54c0925586f8137666a5dfea12efd44; query-on df314c4bb77945ff569af5fde2a2aeef8e0190765ab042c244de3db9c6f78315
• medium: build-off db25c445975caf5e24674629edc5756aec500f0a5bbe4faf95334f42d2874c0b; query-off 0419dbd965a51b2168e424f8a908ecd2c8fdbc661747e45eabf88caa4a64da25; build-on 4b83fce4ee842ef5c9335d5fe9228c5583cd77e2a7a233f98f5019111281b518; query-on 252184ba8473e4ee7ff1506f74fcf0cfeeb378fbb933e4c6a59a08dd0735103f
• large: build-off a0a3249247c4dbefb35311ddac6df0e1fc81aed247f0af0523532022cc6b78bf; query-off af0ce62200a661b9514e1372adfa824ccecf02e75ff86acc53441e63e7fe393c; build-on 066b556726fc521974e8971f074ae42dd962d7fe47cfd9ee9a3cd586da2ce5d9; query-on e3a332da4d3dfdd5d0a30c4f60f496c384415c8d0882edcfa47062da19ddf103

Reproduction

Build the package, prepare each unique fixture outside measurement, then run it:

pnpm --filter @constructive-io/perf-harness build
node packages/perf-harness/dist/scoped-catalog.js prepare --database-url <url> --fixture <unique-cperf-name> --size <small|medium|large>
node packages/perf-harness/dist/scoped-catalog.js run --database-url <url> --fixture <same-name> --size <same-size> --repetitions 10 --seed 20260819 --output-directory packages/perf-harness/artifacts/<name>

Run small before adding medium noise and medium before adding large noise when using one dedicated PostgreSQL server. Fixture preparation never replaces or drops schemas. The large synthetic result demonstrates scale behavior; it is not a claim about typical production gains.

The old cperf tenant-density server, HTTP/open-loop load, p99 qualification, heap ramps, governor/cache/retirement arms, and soak tests were intentionally not migrated.

Validation

Passing:

• graphile-scoped-introspection: 32 tests / 8 suites, including 6 real PostgreSQL dependency-closure cases
• performance harness: 18 tests / 9 suites
• scoped graphile-settings wiring: 3 tests
• CJS and ESM builds for perf-harness and graphile-scoped-introspection
• affected lint with no autofix
• pnpm install --frozen-lockfile
• git diff --check
• PR-wide format-only diff audit; lockfile contains only the required importer entry

No database URL or password appears in the JSON artifacts.