Skip to content

sync: port the machine, repository and image module tables from constructive-db - #120

Merged
pyramation merged 4 commits into
mainfrom
feat/sync-machine-repository-image-modules
Aug 16, 2026
Merged

sync: port the machine, repository and image module tables from constructive-db#120
pyramation merged 4 commits into
mainfrom
feat/sync-machine-repository-image-modules

Conversation

@pyramation

Copy link
Copy Markdown
Contributor

Summary

constructive-db vendors these modules under pgpm-modules/ and edits them in place, so the two trees drift until someone ports the changes back (last done in #119). Two modules had drift: types and metaschema-modules.

types — one new change: upload_ids(upload[]) -> text[]. It exists because an array of uploads has no indexable "which file is this?" expression. A scalar upload column answers that with (value ->> 'id'), which a btree expression index serves; an array cannot, so storage GC and the files→document sync trigger both key off this function, and metaschema_generators.file_ref_field emits a GIN index over it:

CREATE INDEX … USING gin (upload_ids(attachments));
-- GC / sync then ask:  WHERE upload_ids(col) && ARRAY[NEW.id::text]

IMMUTABLE STRICT (an index expression must be immutable) and it drops elements with no id — an upload naming only a url, which the domain allows, is not a reference to a files row, so it contributes nothing rather than a NULL that && would have to reason about.

metaschema-modules — three new module tables the platform now provisions:

  • image_module — the central container image catalog.
  • repository_module — repositories, events, workflows, builds, partitioned build_steps (step + test rows keyed into the build's log object by byte range), local change requests.
  • machine_module — enrolled machines, sessions, and the append-only command ledger.

Plus the backlog that landed on constructive-db main since #119 and was never ported: internal_config_module, k8s_admission_module, the @module_table provenance comments on every module table (what tg_module_install_provenance attributes to an install), and the agent/catalog/resource/route module column additions.

Notes:

  • config_secrets_user_module exists only upstream and is kept (plan line reinserted in place).
  • jwt-claims, metaschema-schema and object-store still carry unported drift from constructive-db; out of scope here.
  • sql/ artifacts regenerated with pgpm package; metaschema-modules snapshots re-recorded against the ported deploy tree; a new upload-ids.test.ts covers ordering, id-less elements, strictness, volatility and the GIN index path.
  • A publish needs a version bump — 0.41.0 is already released and these changes add objects to it.

Link to Devin session: https://app.devin.ai/sessions/45a3052973e44fb2b78bca5a8a161b2e
Requested by: @pyramation

…ructive-db

constructive-db vendors these modules under pgpm-modules/ and edits them in
place, so the two trees drift until someone ports the changes back (last done
in #119). Two modules had drift: types and metaschema-modules.

types gains upload_ids(upload[]) -> text[], the one indexable expression for a
column holding many uploads: a scalar upload column answers "which file is
this?" with (value ->> 'id') and a btree expression index serves it, an array
cannot, so storage garbage collection and the files-to-document sync trigger
both key off this function and file_ref_field emits a GIN index over it.
IMMUTABLE (index expressions must be) and it drops elements with no id, since
an upload naming only a url is not a reference to a files row.

metaschema-modules gains three module tables the platform now provisions --
image_module (central container image catalog), repository_module
(repositories, events, workflows, builds, partitioned build_steps, local
change requests) and machine_module (enrolled machines, sessions, append-only
command ledger) -- plus internal_config_module and k8s_admission_module, and
the @module_table provenance comments and agent/catalog/resource/route column
additions that landed on constructive-db main since #119.

config_secrets_user_module exists only upstream and is kept; jwt-claims,
metaschema-schema and object-store still carry unported drift, out of scope
here. sql/ artifacts regenerated with pgpm package; metaschema-modules
snapshots re-recorded against the ported deploy tree. A publish needs a
version bump -- 0.41.0 is already released and these changes add objects to it.
@pyramation pyramation self-assigned this Aug 16, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

… fixtures

catalog_module gained a NOT NULL redirects_table_id whose uuid_nil() default
cannot satisfy the new FK to metaschema_public."table", same as its sibling
module-table columns, so the fixture inserts must name it.
…re drift from constructive-db

Backlog that predates this sync: current_database_id marked LEAKPROOF, a
derived policy's column_refs persisted on metaschema_public.policy, predicate
calls carrying schema and name separately, and the immutable-objects delete
trigger returning OLD. Artifacts repackaged for the three modules.
@pyramation
pyramation merged commit ab1869c into main Aug 16, 2026
29 checks passed
@pyramation
pyramation deleted the feat/sync-machine-repository-image-modules branch August 16, 2026 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant