Skip to content

feat(meta): report each storage plane's upload/download surface in _meta and add a dynamic storage client - #1765

Merged
pyramation merged 4 commits into
mainfrom
feat/meta-storage-surface-dynamic-client
Aug 19, 2026
Merged

feat(meta): report each storage plane's upload/download surface in _meta and add a dynamic storage client#1765
pyramation merged 4 commits into
mainfrom
feat/meta-storage-surface-dynamic-client

Conversation

@pyramation

@pyramation pyramation commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Summary

PR 2 of the dynamic storage plan (constructive-planning#1779), stacked on #1764. _meta now reports each storage plane's full upload/download surface, and @constructive-io/graphql-query gains a runtime dynamic storage client — so no client (test or app) ever assembles a GraphQL storage name like upload${TypeName} again.

_meta storage block (graphile-meta)

storage { isFilesTable isBucketsTable } expands to the whole plane surface, derived from the same registry-FK pairing (discoverStoragePlanes) and inflection the presigned-url plugin emits from — _meta and the emitted schema cannot disagree:

storage {
  isFilesTable isBucketsTable
  filesType bucketsType        # GraphQL type names of the paired plane
  downloadUrlField             # 'downloadUrl' on the files side, null on buckets
  upload {
    mutation inputType payloadType
    bulkMutation bulkInputType bulkPayloadType
    bulkFileInputType bulkFilePayloadType
    requiresOwnerId            # entity-keyed plane (buckets carry owner_id)
  }
}

buildStorageMeta(codec, build) resolves the codec's plane from the registry facts (cached per-registry); a storage-tagged table that belongs to no discoverable plane throws STORAGE_PLANE_UNPAIRED instead of reporting a partial surface. DOWNLOAD_URL_FIELD is now a single exported constant in graphile-storage-registry, consumed by both the download-url plugin (emitter) and _meta (reporter). discoverStoragePlanes also gains a STORAGE_PLANE_AMBIGUOUS check when two files tables FK one buckets table.

Dynamic storage client (@constructive-io/graphql-query/storage)

Runtime discovery — not codegen/ORM. Three independently usable modules (discovery, document building, orchestration) plus a thin client that composes them; byte-level work is an injected StorageTransport adapter, so graphql-query has no upload/S3 dependency and upload-client supplies the adapter:

import { createStorageClient } from '@constructive-io/graphql-query';
import { storageTransport } from '@constructive-io/upload-client';

const storage = createStorageClient({ execute, transport: storageTransport });
await storage.upload(
  { filesTable: 'files', schemaName: 'storage_public' },  // semantic coordinates
  { file, bucketKey: 'site-content' },
);

Discovery and document building work with no transport at all; upload() without one fails loudly (STORAGE_TRANSPORT_MISSING).

  • resolveStorageSurfaces(_metaResult) groups files/buckets sides into StorageSurface[], throwing STORAGE_META_MALFORMED on duplicate/orphan/inconsistent planes.
  • findStorageSurface(surfaces, selector) resolves exactly one plane by filesTable/schemaName/filesType; not-found and ambiguous selections throw.
  • buildUploadDocument / buildDownloadUrlDocument build documents with gql-ast from the _meta-reported names only (mutation, input type, query.one node field, download field).
  • uploadToSurface(surface, options, { execute, transport }) runs hash → mutation → presigned PUT, refusing entity-keyed planes without ownerId (requiresOwnerId) and skipping the PUT on deduplication. Usable directly when you resolved the surface yourself.
  • upload-client gains storageTransport / createStorageTransport() — a structural adapter over its existing hashFile + putToPresignedUrl, so the dependency stays one-way.
  • convertFromMetaSchema now carries the storage block through meta-object conversion.

The lockfile diff is large because pnpm 10.34.5 rewrote the file into its current single-line resolution format; the only semantic change is the new graphile-meta → graphile-storage-registry workspace link. pnpm install --frozen-lockfile passes.

Next (PR 3, constructive-db): delete the static-site e2e's S3/raw-SQL workaround and migrate it to this client.

Testing

  • pnpm --filter graphile-meta exec jest --runInBand — 134 passed
  • pnpm --filter graphile-storage-registry test — 12 passed
  • pnpm --filter @constructive-io/graphql-query exec jest --runInBand — 40 passed (22 new)
  • pnpm --filter @constructive-io/upload-client test — 24 passed
  • pnpm --filter graphile-presigned-url-plugin test — 69 passed (MinIO up)
  • pnpm lint — clean

Link to Devin session: https://app.devin.ai/sessions/726e066d64e04f29bdbea669c09c6341
Requested by: @pyramation

…in _meta and add a dynamic storage client

_meta's storage block now carries the plane's files/buckets type names,
download-URL field, and the complete upload surface (mutation, input/payload
types, bulk variants, requiresOwnerId), derived from the same registry-FK
pairing and inflection the presigned-url plugin emits from.

@constructive-io/graphql-query gains a runtime storage client that discovers
planes from _meta, resolves a plane by semantic coordinates, builds upload and
download documents with gql-ast, and runs the hash -> mutation -> presigned PUT
orchestration — no caller ever assembles a GraphQL name.
@pyramation pyramation self-assigned this Aug 19, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

Base automatically changed from feat/storage-plane-registry-pairing to main August 19, 2026 03:52
@blacksmith-sh

This comment has been minimized.

@pyramation
pyramation merged commit 96d1c58 into main Aug 19, 2026
20 checks passed
@pyramation
pyramation deleted the feat/meta-storage-surface-dynamic-client branch August 19, 2026 04:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant