From 35004f3aef899584ae7420fe5ad74c13a752b85a Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Fri, 24 Jul 2026 17:23:33 +1000 Subject: [PATCH 1/9] =?UTF-8?q?feat(cli):=20add=20the=20stash-sql=20and=20?= =?UTF-8?q?stash-edge=20skills=20=E2=80=94=20raw-SQL=20predicates=20and=20?= =?UTF-8?q?the=20WASM=20entry?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes #754. No shipped skill covered the integrations that don't use an ORM: hand-written SQL over `pg` / `postgres-js`, and the WASM entry on Deno / Supabase Edge Functions / Workers. Grepping the skills `stash init` installs for `postgres-js|::jsonb::eql|sql.json|query_text_search` returned one hit, in an unrelated code comment — so an integration on that path had to recover the binding surface from `dist/*.d.ts`, the Postgres catalog, and experiment. Two skills rather than one, per the scope question in the issue: the raw-SQL predicate cookbook serves a supported plain-Node path (`hono-pg`) with no edge or WASM involvement, so scoping it under an edge-named skill would leave that surface uncovered. skills/stash-sql — the predicate matrix (which of `=`, `<>`, `<`, `>=`, `@@`, `@>` each column domain accepts, against which `eql_v3.query_*` operand), storage-vs-query payload shapes, per-driver parameter binding, and recipes for equality / free-text / range / ORDER BY / JSON containment / field selectors. skills/stash-edge — import specifier per runtime, the four mandatory `CS_*` variables and minting them with `stash env`, how the WASM client surface differs from the native typed client, and why a schema module can't be shared across the two entries. Both carry the credential-identity rule, also folded into stash-cli (under `env` and `encrypt backfill`, where it bites) and stash-supabase: EQL index terms derive from the ZeroKMS client key, so rows written under one credential and queried under another decrypt correctly and never match a query. Three claims were corrected against a live EQL v3 3.0.2 install rather than carried over from the issue: - The binding rule is driver-specific. On postgres-js a bare object fails to INSERT and `JSON.stringify(...)::jsonb` fails the domain CHECK; only `sql.json()` works in both positions. On `pg` all three forms work. - A bare-`jsonb` operand does not fall through to native jsonb semantics — EQL defines `jsonb` overloads that coerce to the *storage* domain, which requires the ciphertext key `c`, so a query term without the domain cast raises a CHECK violation rather than silently matching nothing. - The schema type incompatibility reproduces in both directions, not one. Also fixes the wasm-inline module JSDoc, which passed `OidcFederationStrategy.create(...)`'s Result straight to `config.authStrategy` without unwrapping — the same JSDoc the raw-SQL surface was being reverse-engineered from. SKILL_MAP (CLI + wizard) installs both for `postgresql` and `supabase`; Drizzle and Prisma Next get cross-links from their own skills instead. --- .changeset/stash-sql-edge-skills.md | 56 +++ AGENTS.md | 4 +- .../init/lib/__tests__/install-skills.test.ts | 22 + .../src/commands/init/lib/install-skills.ts | 17 +- .../cli/src/commands/init/lib/setup-prompt.ts | 4 + packages/stack/src/wasm-inline.ts | 15 +- packages/wizard/src/lib/install-skills.ts | 12 +- skills/stash-cli/SKILL.md | 10 + skills/stash-drizzle/SKILL.md | 2 + skills/stash-edge/SKILL.md | 398 ++++++++++++++++++ skills/stash-encryption/SKILL.md | 7 +- skills/stash-indexing/SKILL.md | 4 +- skills/stash-prisma-next/SKILL.md | 5 +- skills/stash-sql/SKILL.md | 365 ++++++++++++++++ skills/stash-supabase/SKILL.md | 8 + 15 files changed, 918 insertions(+), 11 deletions(-) create mode 100644 .changeset/stash-sql-edge-skills.md create mode 100644 skills/stash-edge/SKILL.md create mode 100644 skills/stash-sql/SKILL.md diff --git a/.changeset/stash-sql-edge-skills.md b/.changeset/stash-sql-edge-skills.md new file mode 100644 index 00000000..1b6d8ca7 --- /dev/null +++ b/.changeset/stash-sql-edge-skills.md @@ -0,0 +1,56 @@ +--- +'stash': minor +'@cipherstash/wizard': minor +'@cipherstash/stack': patch +--- + +Two new bundled agent skills for the integrations that don't use an ORM — +`stash-sql` and `stash-edge` (#754). + +Everything a raw-SQL or edge integration needed was reachable only from +`dist/*.d.ts` JSDoc, the Postgres catalog, or experiment: grepping the skills +`stash init` installs for `postgres-js|::jsonb::eql|sql.json|query_text_search` +returned a single hit, in an unrelated code comment. + +**`stash-sql`** — hand-written SQL over `pg` / `postgres-js`, no ORM. The +column-domain-to-query-domain operator matrix (which of `=`, `<>`, `<`, `>=`, +`@@`, `@>` each encrypted domain accepts, and against which `eql_v3.query_*` +operand), the storage-vs-query payload distinction, per-driver parameter +binding, recipes for equality / free-text / range / `ORDER BY` / JSON +containment / JSON field selectors, and the `information_schema` drift check. +Two failure modes get their mechanism spelled out: pre-stringifying a payload +on postgres-js double-encodes it into a jsonb *string* scalar, tripping the +domain CHECK with a message naming neither JSON nor encoding; and leaving an +operand as bare `jsonb` silently selects a different operator overload — one +that coerces to the *storage* domain and so rejects the ciphertext-free query +term. + +**`stash-edge`** — the `@cipherstash/stack/wasm-inline` entry for Deno, +Supabase Edge Functions, Cloudflare Workers, and Bun. Import specifier per +runtime, the four mandatory `CS_*` variables and minting them with +`stash env`, how the WASM client surface differs from the native typed client +(no `.audit()`, no `.withLockContext()`, per-item bulk shape, ESM-only), and +the auth-strategy `Result` that must be unwrapped before it reaches +`config.authStrategy`. + +Both carry **the credential-identity rule**, a silent data-loss footgun now +also stated in `stash-cli` (under `env` and `encrypt backfill`) and +`stash-supabase`: EQL index terms derive from the ZeroKMS client key, so rows +written under one credential and queried under another decrypt correctly and +never match a query, with no error. + +`stash-encryption` now states that the two entries' schema types **do not +interchange** — their column classes carry private fields, so TypeScript +compares them nominally and rejects a shared schema module in both directions. +It works at runtime, which makes a type assertion the tempting fix; the +guidance is to author the schema against exactly one entry instead. + +`stash init` / `stash impl` handoffs and the `@cipherstash/wizard` skills +prompt install both skills for the `postgresql` and `supabase` integrations. +Drizzle and Prisma Next get cross-links from their own skills instead, since +those integrations emit correctly-typed operands themselves. + +Also fixes the `@cipherstash/stack/wasm-inline` module JSDoc, which showed +`OidcFederationStrategy.create(...)`'s `Result` being passed straight to +`config.authStrategy` without unwrapping — the same JSDoc the raw-SQL surface +was being reverse-engineered from. diff --git a/AGENTS.md b/AGENTS.md index 943dbb98..bd5c17b0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -85,7 +85,7 @@ If these variables are missing, tests that require live encryption will fail or - `e2e/*`: Cross-package end-to-end tests (package managers, supply chain, Prisma example README) - `examples/*`: Working apps (basic, prisma, supabase-worker) - `docs/plans/*`: Internal design plans. User-facing documentation lives at https://cipherstash.com/docs (not in this repo). -- `skills/*`: Agent skills (`stash-cli`, `stash-encryption`, `stash-indexing`, `stash-drizzle`, `stash-dynamodb`, `stash-supabase`, `stash-prisma-next`, `stash-supply-chain-security`) +- `skills/*`: Agent skills (`stash-cli`, `stash-encryption`, `stash-indexing`, `stash-sql`, `stash-edge`, `stash-drizzle`, `stash-dynamodb`, `stash-supabase`, `stash-prisma-next`, `stash-supply-chain-security`) ## Agent Skills — these ship to customers @@ -113,6 +113,8 @@ nothing type-checks them, and the damage lands in a customer's repo, not ours. | Drizzle / Supabase / Prisma Next / DynamoDB integrations | `skills/stash-drizzle`, `skills/stash-supabase`, `skills/stash-prisma-next`, `skills/stash-dynamodb` | | The rollout/cutover lifecycle (`packages/migrate`, `stash encrypt *`) | `skills/stash-encryption` and `skills/stash-cli` | | The `@cipherstash/eql` pin, `eql install`/`eql migration` behaviour, or index-related SQL guidance | `skills/stash-indexing` | +| The EQL operator/domain surface (`eql_v3.query_*` casts, predicate forms) | `skills/stash-sql` | +| `packages/stack/src/wasm-inline.ts`, the WASM entry's exports, or `stash env` | `skills/stash-edge` | | pnpm config, CI workflows, dependency policy | `skills/stash-supply-chain-security` | | The durable agent rules themselves | `packages/cli/src/commands/init/doctrine/AGENTS-doctrine.md` | diff --git a/packages/cli/src/commands/init/lib/__tests__/install-skills.test.ts b/packages/cli/src/commands/init/lib/__tests__/install-skills.test.ts index e4a2e3aa..bc9d6ec4 100644 --- a/packages/cli/src/commands/init/lib/__tests__/install-skills.test.ts +++ b/packages/cli/src/commands/init/lib/__tests__/install-skills.test.ts @@ -82,6 +82,28 @@ describe('SKILL_MAP', () => { expect(SKILL_MAP.postgresql).not.toContain('stash-supabase') expect(SKILL_MAP.postgresql).not.toContain('stash-prisma-next') }) + + // #754: the no-ORM path had no source for the raw-SQL binding surface or + // the WASM entry — an integration on it had to reverse-engineer both from + // `dist/*.d.ts` and the Postgres catalog. `postgresql` is that path; + // Supabase shares it (Edge Functions are the flagship WASM-entry use, and + // its migrations/RPC are hand-written SQL). + it.each([ + 'postgresql', + 'supabase', + ] as const)('%s includes the raw-SQL and edge skills', (integration) => { + expect(SKILL_MAP[integration]).toContain('stash-sql') + expect(SKILL_MAP[integration]).toContain('stash-edge') + }) + + // The ORM integrations emit correctly-typed operands themselves, so they + // get cross-links from their own skills rather than the full install. + it.each([ + 'drizzle', + 'prisma-next', + ] as const)('%s does not install the raw-SQL skill', (integration) => { + expect(SKILL_MAP[integration]).not.toContain('stash-sql') + }) }) describe('skillsFor', () => { diff --git a/packages/cli/src/commands/init/lib/install-skills.ts b/packages/cli/src/commands/init/lib/install-skills.ts index acd15818..a57897d4 100644 --- a/packages/cli/src/commands/init/lib/install-skills.ts +++ b/packages/cli/src/commands/init/lib/install-skills.ts @@ -12,10 +12,16 @@ import { findBundledDir } from './bundled-paths.js' */ export const SKILL_MAP: Record = { drizzle: ['stash-encryption', 'stash-drizzle', 'stash-indexing', 'stash-cli'], + // Supabase gets the raw-SQL and edge skills on top of its own: Edge + // Functions are the flagship use of the WASM entry, and Supabase projects + // write hand-written SQL in migrations and RPC even when the app itself + // goes through PostgREST (#754). supabase: [ 'stash-encryption', 'stash-supabase', 'stash-indexing', + 'stash-sql', + 'stash-edge', 'stash-cli', ], 'prisma-next': [ @@ -24,7 +30,16 @@ export const SKILL_MAP: Record = { 'stash-indexing', 'stash-cli', ], - postgresql: ['stash-encryption', 'stash-indexing', 'stash-cli'], + // The no-ORM path: `stash-sql` (binding + predicate forms) and `stash-edge` + // (WASM entry, CS_* credentials) are the two skills this integration has no + // other source for — everything else assumes an ORM emits the operands. + postgresql: [ + 'stash-encryption', + 'stash-indexing', + 'stash-sql', + 'stash-edge', + 'stash-cli', + ], } /** The skills every integration gets — the safe fallback for an unmapped one. */ diff --git a/packages/cli/src/commands/init/lib/setup-prompt.ts b/packages/cli/src/commands/init/lib/setup-prompt.ts index dd5ba0de..b3aefdbb 100644 --- a/packages/cli/src/commands/init/lib/setup-prompt.ts +++ b/packages/cli/src/commands/init/lib/setup-prompt.ts @@ -110,6 +110,10 @@ const SKILL_PURPOSES: Record = { 'Prisma Next-specific patterns: `cipherstash.*` field constructors, migration flow, encrypted query operators', 'stash-indexing': 'index recipes for encrypted columns — the `eql_v3` extractor functional indexes, Supabase/managed-Postgres constraints, EXPLAIN verification', + 'stash-sql': + 'hand-written SQL over `pg` / `postgres-js`: the encrypted predicate matrix, `eql_v3.query_*` operand casts, per-driver parameter binding', + 'stash-edge': + 'the `@cipherstash/stack/wasm-inline` entry for Deno / Supabase Edge Functions / Workers: imports, `CS_*` credentials, the credential-identity rule', 'stash-dynamodb': 'DynamoDB encryption: per-item encrypt/decrypt, HMAC attribute keys, audit logging', 'stash-cli': diff --git a/packages/stack/src/wasm-inline.ts b/packages/stack/src/wasm-inline.ts index 0b7e8966..54c79138 100644 --- a/packages/stack/src/wasm-inline.ts +++ b/packages/stack/src/wasm-inline.ts @@ -64,17 +64,24 @@ * import { OidcFederationStrategy } from "@cipherstash/stack/wasm-inline" * import { cookieStore } from "@cipherstash/auth/cookies" * - * const authStrategy = OidcFederationStrategy.create( + * // `create` returns a Result — UNWRAP it. `config.authStrategy` takes the + * // strategy itself; handing it the Result fails later and opaquely. + * const strategy = OidcFederationStrategy.create( * "crn:ap-southeast-2.aws:my-workspace-id", () => getClerkSessionToken(req), * { store: cookieStore({ request: req, responseHeaders }) }, * ) - * const client = await Encryption({ schemas, config: { authStrategy, clientId, clientKey } }) + * if (strategy.failure) throw new Error(strategy.failure.error.message) + * + * const client = await Encryption({ + * schemas, config: { authStrategy: strategy.data, clientId, clientKey }, + * }) * ``` * * For service-to-service / CI use with a custom token store, build an * `AccessKeyStrategy.create(workspaceCrn, accessKey, { store })` the same - * way (it derives the region from the CRN). Both strategies are - * re-exported from this entry. + * way — same Result-unwrapping, and it derives the region from the CRN. Both + * strategies are re-exported from this entry. An auth strategy and + * `config.accessKey` are mutually exclusive. */ import { withResult } from '@byteslice/result' diff --git a/packages/wizard/src/lib/install-skills.ts b/packages/wizard/src/lib/install-skills.ts index 474bc2bd..25442278 100644 --- a/packages/wizard/src/lib/install-skills.ts +++ b/packages/wizard/src/lib/install-skills.ts @@ -13,14 +13,24 @@ import type { Integration } from './types.js' */ const SKILL_MAP: Record = { drizzle: ['stash-encryption', 'stash-drizzle', 'stash-indexing', 'stash-cli'], + // `stash-sql` / `stash-edge` mirror the CLI's SKILL_MAP — see the comments + // there for why Supabase and the generic (no-ORM) path get them (#754). supabase: [ 'stash-encryption', 'stash-supabase', 'stash-indexing', + 'stash-sql', + 'stash-edge', 'stash-cli', ], prisma: ['stash-encryption', 'stash-indexing', 'stash-cli'], - generic: ['stash-encryption', 'stash-indexing', 'stash-cli'], + generic: [ + 'stash-encryption', + 'stash-indexing', + 'stash-sql', + 'stash-edge', + 'stash-cli', + ], } /** diff --git a/skills/stash-cli/SKILL.md b/skills/stash-cli/SKILL.md index aa1c03c2..4113b0a0 100644 --- a/skills/stash-cli/SKILL.md +++ b/skills/stash-cli/SKILL.md @@ -457,6 +457,8 @@ Backfill **detects a `public.eql_v3_*` target column as EQL v3** from its Postgr **Dual-write precondition.** The application must already write both `` and `_encrypted` on every insert and update. Otherwise rows written *during* the backfill land in plaintext only, silently. The first run prompts (interactive) or requires `--confirm-dual-writes-deployed` (non-interactive), then records `dual_writing`. Resumes don't re-prompt. +**Credential precondition — run the backfill with the *application's* credentials.** Backfill encrypts through whichever `CS_*` credentials are in its environment, and EQL index terms derive from the ZeroKMS client key. Backfilling from a laptop on the local device profile, then querying from an app using credentials minted by `stash env`, produces rows that decrypt correctly and **never match a query** — with no error. Export the target environment's `CS_*` values in the shell running the backfill. See [`env`](#env) and `stash-edge` § The Credential-Identity Rule. + | Flag | Description | |---|---| | `--table` / `--column` | Required | @@ -516,6 +518,14 @@ CS_CLIENT_KEY= CS_CLIENT_ACCESS_KEY=CSAK… ``` +> **Every writer of a searchable column must use these same credentials** — +> including `stash encrypt backfill`, seed scripts, and admin tools — or their +> rows decrypt but never match a query. EQL index terms derive from the ZeroKMS +> client key, so a row written under one credential and queried under another +> decrypts correctly and silently fails every search. Mint one credential per +> environment and export it for **every** process that writes that +> environment's data. See `stash-edge` § The Credential-Identity Rule. + Things to know: - **The access key is shown exactly once** — CTS cannot re-reveal it. Pipe the diff --git a/skills/stash-drizzle/SKILL.md b/skills/stash-drizzle/SKILL.md index 50ae36fd..8a4286d2 100644 --- a/skills/stash-drizzle/SKILL.md +++ b/skills/stash-drizzle/SKILL.md @@ -417,6 +417,8 @@ import { index } from "drizzle-orm/pg-core" Run `ANALYZE ` after the migration applies — an expression index gathers no statistics at `CREATE INDEX` time. For when to create indexes during a rollout (after backfill, before switching reads), engagement rules, and `EXPLAIN` verification, see the `stash-indexing` skill. +> **Dropping to raw SQL?** `db.execute(sql\`…\`)` bypasses the operators this integration emits, so you own the operand casts yourself — an encrypted predicate needs its needle cast to the column's `eql_v3.query_*` domain, and the driver's parameter-binding rules differ between `pg` and `postgres-js`. The `stash-sql` skill is the reference for both. + ## Migrating an Existing Column to Encrypted The hard case: a Drizzle table that already exists in production with live data in a plaintext column you want to encrypt. You can't just change the column type — that would drop the data and break NOT NULL constraints. diff --git a/skills/stash-edge/SKILL.md b/skills/stash-edge/SKILL.md new file mode 100644 index 00000000..2359d39f --- /dev/null +++ b/skills/stash-edge/SKILL.md @@ -0,0 +1,398 @@ +--- +name: stash-edge +description: Run CipherStash encryption on edge and non-Node runtimes with the `@cipherstash/stack/wasm-inline` entry — Deno, Supabase Edge Functions, Cloudflare Workers, and Bun. Covers the import specifier per runtime, the four mandatory `CS_*` variables and minting them with `stash env`, the credential-identity rule (rows written under different credentials decrypt but never match a query), how the WASM client surface differs from the native typed client, and why an EQL v3 schema module cannot be shared across the two entries. Use when adding encryption to a Supabase Edge Function, a Worker, or a Deno service; when a native module fails to load in a deployed runtime; when wiring `CS_*` secrets into an edge deploy; or when encrypted search returns zero rows on the edge but works locally. +--- + +# Encryption on the Edge (WASM entry) + +`@cipherstash/stack` has two runtime entries. The default one binds +`@cipherstash/protect-ffi`, a Node-API native module, and must be loaded by +Node's own `require`. **`@cipherstash/stack/wasm-inline` is the entry for +everywhere else** — it carries the WASM build of the same engine as a base64 +blob inside the JS, so there is no native binding, no separate `.wasm` fetch, +and nothing for a bundler to externalise. + +This skill covers that entry and the deployment shape around it. It is EQL v3 +throughout. For the SQL that actually queries the encrypted columns — the +predicate forms and driver binding rules — see `stash-sql`; edge functions +almost always talk to Postgres over a raw driver, so the two are usually read +together. + +## When to Use This Skill + +- Adding encryption to a Supabase Edge Function, Cloudflare Worker, Deno + service, or Bun app. +- A deployed runtime fails to load the native module (`protect-ffi`), or a + bundler chokes trying to include it. +- Wiring `CS_*` credentials into an edge deploy, or minting them at all. +- Encrypted search works locally but returns **zero rows** in the deployed + function — see [The Credential-Identity Rule](#the-credential-identity-rule-a-silent-data-footgun). +- A schema module shared with Node tooling fails to typecheck against the + edge client. + +## Choosing the Entry + +| Runtime | Entry | Why | +|---|---|---| +| Node server, Next.js server code | `@cipherstash/stack` (+ `/v3`) | Native NAPI is faster; keep `@cipherstash/protect-ffi` external (e.g. Next's `serverExternalPackages`) | +| Supabase Edge Functions | `@cipherstash/stack/wasm-inline` | Deno, V8-only, no native modules | +| Cloudflare Workers | `@cipherstash/stack/wasm-inline` | V8 isolate, no native modules | +| Deno (any) | `@cipherstash/stack/wasm-inline` | No NAPI under Deno's default permissions | +| Bun | `@cipherstash/stack/wasm-inline` | Works, and avoids native-module resolution differences | +| Anywhere bundling server code | `@cipherstash/stack/wasm-inline` | Bundles cleanly; nothing to externalise | + +**The WASM entry is ESM-only.** Its `exports` map has an `import` condition +and no `require` — deliberately, since the runtimes it targets are ESM. A CJS +`require('@cipherstash/stack/wasm-inline')` will not resolve. Node consumers +that need it must be ESM (`"type": "module"` or `.mjs`). + +## Importing It + +### Supabase Edge Functions / Deno — `npm:` specifier + +The Edge runtime resolves `npm:` specifiers at function start; there is no +build step. + +```ts +import { + Encryption, encryptedTable, types, isEncrypted, +} from 'npm:@cipherstash/stack@1.0.0-rc.4/wasm-inline' +``` + +**Pin an exact version.** Deno caches by specifier, so an unpinned import +drifts between deploys. And while the package is on a prerelease line, a +caret range does not do what it looks like: `@^1.0.0` will **not** match +`1.0.0-rc.4`, because semver ranges exclude prereleases unless the range +itself names one. Use the exact version, or a prerelease-bearing range +(`@^1.0.0-rc.4`). Check what is current with `npm view @cipherstash/stack dist-tags`. + +### Deno with an import map + +For a project with a `deno.json`, map the specifier once and import the bare +name everywhere: + +```jsonc +{ + "imports": { + "@cipherstash/stack/wasm-inline": "npm:@cipherstash/stack@1.0.0-rc.4/wasm-inline" + } +} +``` + +```ts +import { Encryption, encryptedTable, types } from '@cipherstash/stack/wasm-inline' +``` + +> **No `--allow-ffi` needed.** The whole point of this entry is that nothing +> native loads. If a Deno process running this entry ever demands an FFI +> permission, something has resolved the native entry instead — check the +> import path before granting anything. + +### Cloudflare Workers / Bun / bundlers — normal install + +```bash +npm install @cipherstash/stack +``` + +```ts +import { Encryption, encryptedTable, types } from '@cipherstash/stack/wasm-inline' +``` + +No `externals`, no `nodeExternals`, no `serverExternalPackages` entry. If a +build config already externalises `@cipherstash/protect-ffi` for the native +entry, that config does not apply here and can be left alone. + +## Credentials + +The edge client takes **all four** `CS_*` values explicitly. There is no +credential discovery: `~/.cipherstash` does not exist in a Worker or an Edge +Function container, and there is no device-code login to fall back on. + +```ts +const client = await Encryption({ + schemas: [users], + config: { + workspaceCrn: Deno.env.get('CS_WORKSPACE_CRN')!, + accessKey: Deno.env.get('CS_CLIENT_ACCESS_KEY')!, + clientId: Deno.env.get('CS_CLIENT_ID')!, + clientKey: Deno.env.get('CS_CLIENT_KEY')!, + }, +}) +``` + +Read them from the platform's environment accessor — `Deno.env.get(...)` on +Deno/Supabase, the `env` binding argument on Workers, `process.env` on Bun. + +### Minting them: `stash env` + +```bash +stash env --name my-app-prod # print the four vars to stdout +stash env --name my-app-prod --write # write .env.production.local (mode 0600) +stash env --name edge-dev --write .env.local +``` + +This creates a fresh ZeroKMS client **and** a CipherStash access key from your +local `stash auth login` session. Things that matter here: + +- **The access key is shown exactly once.** Pipe it straight into the secret + store; it cannot be re-revealed. +- **Stdout is pipe-clean** — only the dotenv block goes to stdout, so + `stash env --name x > prod.env` and pipes into secret-store CLIs are safe. +- Each run mints a **new** credential, and duplicate names are rejected. Use a + distinct `--name` per environment. +- `CS_CLIENT_KEY` and `CS_CLIENT_ACCESS_KEY` are secrets. Never commit them; + put placeholder names in `.env.example` instead. + +### Getting them into the runtime + +```bash +# Supabase — local +supabase functions serve --env-file .env.local my-function + +# Supabase — deployed +stash env --name my-app-prod --write .env.production.local +supabase secrets set --env-file .env.production.local + +# Cloudflare Workers +wrangler secret put CS_CLIENT_KEY # repeat per variable + +# Vercel / other platforms +vercel env add CS_CLIENT_KEY production +``` + +## The Credential-Identity Rule (a silent data footgun) + +> **Every writer of a searchable column must use the same credentials as every +> reader — including `stash encrypt backfill`, seed scripts, and admin tools. +> Rows written under different credentials decrypt correctly but never match a +> query.** + +EQL's searchable-encryption index terms (the `hm`, `op`, `bf` fields in the +stored payload) derive from the **ZeroKMS client key**, not from the workspace +or keyset. Two clients in the same workspace with different `CS_CLIENT_ID` / +`CS_CLIENT_KEY` pairs therefore produce **different terms for the same +plaintext**. + +The consequences are asymmetric, which is what makes this hard to spot: + +- **Decryption still works.** The data key is wrapped through ZeroKMS against + the workspace, so any authorised client in the workspace can decrypt the + row. Round-trip tests pass. +- **Search silently fails.** An equality or match predicate compares the + query term against the stored term. Different client keys, different terms, + no match — and no error. The query returns zero rows exactly as though the + data were absent. + +The classic way to hit it: run `stash encrypt backfill` from a laptop (using +the local device-profile credentials), then query those rows from an Edge +Function using `CS_*` values minted by `stash env`. Every row decrypts. No +search ever matches. + +**What to do:** + +- Mint one credential per *environment*, and use it for **every** process that + touches that environment's data — the app, the backfill, seed scripts, admin + jobs, and one-off scripts alike. +- Before running `stash encrypt backfill` against an environment, export that + environment's `CS_*` values into the shell running it. +- If rows have already been written under the wrong credentials, re-encrypt + them with the correct client: read (decryption still works), then write back + through a client built with the target credentials. + +**Diagnosing it:** if `decrypt` returns the right plaintext but an equality +query on the same row returns nothing, compare the stored term against a +freshly minted one for the same plaintext. Matching plaintext with differing +`hm` values is this bug, not an indexing problem. + +## The Client Surface + +`Encryption` from the WASM entry. **Both entries name the factory +`Encryption`**, so the import path is the only thing that distinguishes them — +which makes a stray `import { Encryption } from '@cipherstash/stack'` easy to +miss and confusing to debug, because the two clients take different config and +different bulk shapes. Check the specifier first whenever an edge client +behaves unexpectedly. + +Every fallible method returns the same `{ data } | { failure }` Result +contract as the native client; unwrap before use. + +```ts +const enc = await client.encrypt('alice@example.com', { table: users, column: users.email }) +if (enc.failure) throw new Error(enc.failure.message) + +const dec = await client.decrypt(enc.data) +if (dec.failure) throw new Error(dec.failure.message) +``` + +Available: `encrypt`, `decrypt`, `isEncrypted`, `encryptQuery`, +`encryptQueryBulk`, `bulkEncrypt`, `bulkDecrypt`, `encryptModel`, +`decryptModel`, `bulkEncryptModels`, `bulkDecryptModels`. + +### How it differs from the native typed client + +| | Native (`@cipherstash/stack`) | WASM (`@cipherstash/stack/wasm-inline`) | +|---|---|---| +| Factory | `Encryption({ schemas })` | `Encryption({ schemas, config })` — same name, different module | +| Schema authoring | `encryptedTable` / `types` from `@cipherstash/stack/v3` | the entry's own re-exports (see below) | +| Config | discovered from env / `~/.cipherstash` | all four `CS_*` passed explicitly | +| Typing | signatures derived from the schema | schema-aware, but not the full typed client | +| `.audit()` | chainable on operations | **not available** | +| `.withLockContext()` | chainable on operations | **not available** — see below | +| `bulkEncrypt` shape | `(plaintexts, { table, column })`, `{ id, plaintext }` envelopes | per-item `{ plaintext, table, column }`, plain index-aligned array | +| Module format | ESM + CJS | **ESM only** | + +**Identity-bound encryption is configured, not chained.** There is no +`.withLockContext()` on this entry. Build an `OidcFederationStrategy` (or +`AccessKeyStrategy` for service-to-service) and pass it as +`config.authStrategy`, so the client is authenticated *as the end user* for +its whole lifetime: + +```ts +import { Encryption, OidcFederationStrategy } from '@cipherstash/stack/wasm-inline' + +// `create` returns a Result — unwrap it. Passing the Result itself as +// `authStrategy` is the easy mistake, and it fails opaquely later. +const strategy = OidcFederationStrategy.create( + workspaceCrn, // 'crn::' + () => getUserJwt(req), // called on every re-federation — Clerk, Supabase Auth, … +) +if (strategy.failure) throw new Error(strategy.failure.error.message) + +const client = await Encryption({ + schemas: [users], + config: { authStrategy: strategy.data, clientId, clientKey }, +}) +``` + +`AccessKeyStrategy.create(workspaceCrn, accessKey)` has the same +Result-returning shape, for service-to-service use with a custom token store. +When you pass an auth strategy, do **not** also pass `config.accessKey` — they +are mutually exclusive and the client rejects the combination. + +Construct a client **per request** when using a user-scoped strategy — a +module-level client would bind whichever user happened to arrive first. + +### The bulk shape differs — don't copy the native form + +```ts +// WASM entry: each entry carries its own table and column. +const out = await client.bulkEncrypt([ + { plaintext: 'a@example.com', table: users, column: users.email }, + { plaintext: 'b@example.com', table: users, column: users.email }, +]) +if (out.failure) throw new Error(out.failure.message) +// out.data is index-aligned; a null/undefined plaintext yields null at that index. +``` + +The model helpers (`encryptModel` / `decryptModel` and their bulk forms) *are* +present on this entry and behave as they do natively — declared columns +encrypted by JS property name, everything else passing through, one ZeroKMS +round trip per call. + +## Schema Modules Do Not Cross Entries + +A schema authored with `@cipherstash/stack/v3` **will not typecheck** +against the WASM entry's `Encryption`, and the reverse fails too: + +``` +Type 'EncryptedTextSearchColumn' is not assignable to type 'AnyEncryptedV3Column'. + Types have separate declarations of a private property 'columnName'. +``` + +The two entries ship independent type bundles, and the column classes carry +private fields — which TypeScript compares **nominally**. The declarations are +identical in shape but not the same declaration, so assignment is rejected in +both directions. + +It works fine at runtime, which is the trap: the tempting fix is +`as never` / `as any` on the schema, which silences a real signal and will +keep silencing it after a genuine schema mismatch appears. + +**Author the schema module against exactly one entry, and use that entry's +client with it.** For a project whose encryption runs on the edge, that means +importing `encryptedTable` and `types` from `@cipherstash/stack/wasm-inline` +in the shared schema module: + +```ts +// schema.ts — the single source of truth for this project's schema +import { encryptedTable, types } from '@cipherstash/stack/wasm-inline' + +export const users = encryptedTable('users', { + email: types.TextSearch('email'), + ssn: types.TextEq('ssn'), +}) +``` + +Node-side code that imports this module must then also build its client from +`@cipherstash/stack/wasm-inline` (which runs on Node perfectly well, just with +the WASM engine rather than the native one) and must be ESM. + +If a project genuinely needs the native client on the server *and* the WASM +client on the edge, keep two schema modules and treat their agreement as +something to test, not something the type system will enforce for you. Column +names and domains must match exactly — they are what the database and the +stored payload's `i` identifier are keyed by. + +## Querying from the Edge + +Edge functions rarely have an ORM, so encrypted search is usually hand-written +SQL over `pg` or `postgres-js`. Mint the search needle with `encryptQuery`, +then bind it as a typed parameter: + +```ts +const term = await client.encryptQuery('alice@example.com', { + table: users, column: users.email, queryType: 'equality', +}) +if (term.failure) throw new Error(term.failure.message) + +// postgres-js — bind the unwrapped term, not the Result +const rows = await sql` + SELECT * FROM users + WHERE email = ${sql.json(term.data)}::jsonb::eql_v3.query_text_eq` +``` + +The predicate forms, the per-driver binding rules, and the query-domain names +are the subject of `stash-sql` — read it before writing the first query. The +two rules that bite immediately: the operand must be **cast to the column's +`eql_v3.query_*` domain**, and on `postgres-js` payloads must be bound with +`sql.json(...)`, never pre-stringified. + +## Troubleshooting + +**`Dynamic require of "..." is not supported` / a native `.node` file in the bundle** +— the native entry got imported. Check every import path resolves to +`@cipherstash/stack/wasm-inline`, including transitive ones from your own +shared modules. + +**`require(...) is not a function` / the specifier won't resolve in CJS** — this +entry is ESM-only. Move the consumer to ESM. + +**Missing `CS_*` at runtime** — the secret store was never populated, or the +function was served without `--env-file`. Validate all four at handler entry +and return an actionable error rather than letting client construction fail +opaquely; the example in `examples/supabase-worker` does exactly this. + +**Encryption works, search returns zero rows** — the credential-identity rule +above. Second most likely: a missing index (see `stash-indexing`) makes it +slow, not empty, so empty results point at credentials or an untyped operand +(`stash-sql`). + +**Search needle rejected** — free-text needles must be at least 3 characters; +shorter ones tokenize to nothing. + +**Cold-start latency** — the inlined WASM module is compiled on first use. +Construct the client at module scope when the auth strategy is not +user-scoped, so it is reused across invocations on a warm isolate. + +## Reference + +- `stash-sql` — the raw-SQL predicate cookbook and driver binding rules. +- `stash-encryption` — schema authoring, the `types.*` domain catalog, and the + rollout/cutover lifecycle. +- `stash-cli` — `stash env`, `stash eql install`, `stash encrypt backfill`. +- `stash-indexing` — indexes on encrypted columns (the DDL is the same + wherever the app runs). +- `stash-supabase` — the PostgREST wrapper, for Supabase apps that are not + writing raw SQL. +- Working example: `examples/supabase-worker` in the `cipherstash/stack` repo. +- Bundling guide: https://cipherstash.com/docs/stack/deploy/bundling diff --git a/skills/stash-encryption/SKILL.md b/skills/stash-encryption/SKILL.md index 3c780c1c..02e0a78e 100644 --- a/skills/stash-encryption/SKILL.md +++ b/skills/stash-encryption/SKILL.md @@ -190,7 +190,7 @@ The SDK never logs plaintext data. | `@cipherstash/stack/types` | All TypeScript types | | `@cipherstash/stack-drizzle` | Drizzle ORM integration for EQL v3 schemas — the package root, EQL v3 only (see the `stash-drizzle` skill) | | `@cipherstash/stack-supabase` | `encryptedSupabase` wrapper for Supabase — EQL v3 only (see the `stash-supabase` skill) | -| `@cipherstash/stack/wasm-inline` | The **edge** entry — Deno, Bun, Cloudflare Workers, Supabase Edge Functions. Its own `Encryption` factory plus the v3 authoring surface, `EncryptionErrorTypes`, and the WASM build of protect-ffi inlined into the bundle. No native binding, so no bundler externalisation needed. | +| `@cipherstash/stack/wasm-inline` | The **edge** entry — Deno, Bun, Cloudflare Workers, Supabase Edge Functions. Its own `Encryption` factory plus its own copy of the v3 authoring surface, `EncryptionErrorTypes`, and the WASM build of protect-ffi inlined into the bundle. No native binding, so no bundler externalisation needed. **ESM-only, and its schema types do not interchange with the other entries'** — see the `stash-edge` skill. | | `@cipherstash/stack/dynamodb` | `encryptedDynamoDB` — encrypt/write is **EQL v3 only** (`types.*`); decrypt still reads existing v2 items. See the `stash-dynamodb` skill | | `@cipherstash/stack/schema`, `@cipherstash/stack/client`, `@cipherstash/stack/encryption` | Legacy v2 schema builders and client surface — see "Legacy: EQL v2" below | @@ -431,6 +431,9 @@ for (const item of decrypted.data) { > [!IMPORTANT] > The `client` below is a **different client** from the one used everywhere else in this skill. The edge entry has its own `Encryption` factory — the native `Encryption` client's `bulkEncrypt` takes `(plaintexts, { table, column })` and will fail at runtime if given the per-item shape below. Construct the WASM client explicitly: +> [!IMPORTANT] +> **The schema is not shareable between entries either.** Note that `encryptedTable` and `types` are imported *from the WASM entry* below, not from `@cipherstash/stack/eql/v3`. The entries ship independent type bundles whose column classes carry private fields, so TypeScript compares them **nominally**: a schema authored on one entry is rejected by the other's client, in both directions (`Types have separate declarations of a private property 'columnName'`). It works at runtime, which makes `as any` the tempting fix — don't. Author the shared schema module against exactly one entry and build that entry's client from it. See the `stash-edge` skill. + ```typescript // Deno / Workers / Supabase Edge Functions — note the import path import { Encryption, encryptedTable, types } from "@cipherstash/stack/wasm-inline" @@ -567,7 +570,7 @@ All values in the array must be non-null. ### On the Wire: Operators and Ordering -Scalar filters compare through each domain's `eql_v3.*` operators (`col = term`, `col > term`, ...), and `ORDER BY` on an encrypted column goes through the ordering extractors — `eql_v3.ord_term(col)` for OPE-backed (`Ord`/`Search`) domains, `eql_v3.ord_term_ore(col)` for `OrdOre`. The Drizzle v3 integration emits all of this for you (including `asc`/`desc`, which emit `ORDER BY eql_v3.ord_term(col)`). Over Supabase/PostgREST, the adapter's `order()` works on OPE-backed ordering columns (plain `*_ord`, `text_ord`, `text_search`) by sorting on the column's `col->op` term; `OrdOre`-flavour (`*_ord_ore`) domains and columns with no ordering term are rejected. See the `stash-drizzle` and `stash-supabase` skills. These same extractor expressions are also what you index — the functional-index recipes are in the `stash-indexing` skill. +Scalar filters compare through each domain's `eql_v3.*` operators (`col = term`, `col > term`, ...), and `ORDER BY` on an encrypted column goes through the ordering extractors — `eql_v3.ord_term(col)` for OPE-backed (`Ord`/`Search`) domains, `eql_v3.ord_term_ore(col)` for `OrdOre`. The Drizzle v3 integration emits all of this for you (including `asc`/`desc`, which emit `ORDER BY eql_v3.ord_term(col)`). Over Supabase/PostgREST, the adapter's `order()` works on OPE-backed ordering columns (plain `*_ord`, `text_ord`, `text_search`) by sorting on the column's `col->op` term; `OrdOre`-flavour (`*_ord_ore`) domains and columns with no ordering term are rejected. See the `stash-drizzle` and `stash-supabase` skills. These same extractor expressions are also what you index — the functional-index recipes are in the `stash-indexing` skill. Writing the SQL by hand instead (no ORM, `pg` / `postgres-js`)? The predicate matrix, the `eql_v3.query_*` operand casts, and the per-driver parameter-binding rules are in the `stash-sql` skill; running on Deno / Workers / Supabase Edge Functions, the `stash-edge` skill. ## Encrypted JSON (`types.Json`) diff --git a/skills/stash-indexing/SKILL.md b/skills/stash-indexing/SKILL.md index 5e7b46cc..8ed63bf3 100644 --- a/skills/stash-indexing/SKILL.md +++ b/skills/stash-indexing/SKILL.md @@ -257,7 +257,7 @@ Index not being used: - **Drizzle** — `encryptedIndexes(t)` from `@cipherstash/stack-drizzle` derives the recommended indexes for every encrypted column in the table, or declare individual expression indexes in the schema DSL. See `stash-drizzle` § Indexing Encrypted Columns. - **Prisma Next** — Prisma's schema language cannot express functional indexes; the DDL goes in a migration in the adapter's flow. See `stash-prisma-next`. - **Supabase** — a `supabase/migrations/` file; no superuser needed (see above). See `stash-supabase`. -- **Raw SQL / plain PostgreSQL** — the recipes in this skill, in whatever migration tool owns the schema. Never ad-hoc in production. +- **Raw SQL / plain PostgreSQL** — the recipes in this skill, in whatever migration tool owns the schema. Never ad-hoc in production. The predicates those indexes serve are in `stash-sql`. ## When to Create Indexes During an Encryption Rollout @@ -271,3 +271,5 @@ Index not being used: - `stash-encryption` — the `types.*` domain catalog, wire-format operators and ordering, and the rollout/cutover lifecycle. - `stash-cli` — `stash eql install`, `stash db validate` (its "No indexes on an encrypted column" Info finding is resolved by this skill), `stash encrypt backfill` / `cutover`. - `stash-drizzle`, `stash-supabase`, `stash-prisma-next` — per-integration query patterns; index DDL placement per the section above. +- `stash-sql` — the hand-written predicate forms these indexes serve (`pg` / `postgres-js`, no ORM). +- `stash-edge` — the WASM entry, for apps whose queries run on Deno / Workers / Supabase Edge Functions. diff --git a/skills/stash-prisma-next/SKILL.md b/skills/stash-prisma-next/SKILL.md index 632af947..11e8b448 100644 --- a/skills/stash-prisma-next/SKILL.md +++ b/skills/stash-prisma-next/SKILL.md @@ -178,7 +178,10 @@ The `ANALYZE` is part of the recipe — an expression index has no statistics until it runs. Works as a non-superuser role (Supabase included); only the ORE-flavour (`_ord_ore`) ordering opclass is superuser-gated. For the full model — which domains take which index, engagement rules, `EXPLAIN` -verification, rollout timing — see the `stash-indexing` skill. +verification, rollout timing — see the `stash-indexing` skill. For encrypted +predicates written as raw SQL rather than through the `cipherstash:*` +operators — operand casts to `eql_v3.query_*`, per-driver parameter binding — +see the `stash-sql` skill. In a migration, the recipes ride a raw-SQL operation (`rawSql` from `@prisma-next/postgres/migration`) in the migration's `operations`: diff --git a/skills/stash-sql/SKILL.md b/skills/stash-sql/SKILL.md new file mode 100644 index 00000000..3846b3d5 --- /dev/null +++ b/skills/stash-sql/SKILL.md @@ -0,0 +1,365 @@ +--- +name: stash-sql +description: Query EQL v3 encrypted columns from hand-written SQL over `pg` (node-postgres) or `postgres` (postgres-js) — no ORM. Covers the column-domain-to-query-domain operator matrix (which of `=`, `<>`, `<`, `>=`, `@@`, `@>` each encrypted domain accepts), minting search needles with `encryptQuery`, the per-driver parameter-binding rules for encrypted payloads, and the double-encoding failure that trips the domain CHECK with a message naming neither JSON nor encoding. Use when writing INSERT/SELECT against an encrypted column without an ORM, when a predicate returns zero rows or raises "operator does not exist", or when a domain CHECK constraint rejects an encrypted value on write. +--- + +# Raw SQL Against Encrypted Columns (EQL v3) + +An EQL v3 encrypted column is a **Postgres domain over `jsonb`** (`public.eql_v3_text_search`, +`public.eql_v3_bigint_ord`, …). Reading and writing it from raw SQL is two rules: + +1. **Writing** — bind the `Encrypted` payload your client produced as a + *JSON object* parameter. How you do that differs per driver, and getting it + wrong trips a domain CHECK with an unhelpful message. +2. **Querying** — never send a plaintext. Mint a **query term** with + `encryptQuery` and cast it to the column's matching `eql_v3.query_*` + domain. That cast is what selects the right operator overload: leave the + operand as bare `jsonb` and you get a *different* overload, one that + expects a full storage envelope. + +This covers the `pg` and `postgres-js` drivers with no ORM — plain Node +services, Hono, edge functions. If you use Drizzle, Prisma Next, or the +Supabase client, those integrations emit correct operands for you: see +`stash-drizzle`, `stash-prisma-next`, `stash-supabase` instead. + +## When to Use This Skill + +- Writing `INSERT` / `UPDATE` / `SELECT` against an encrypted column by hand. +- A predicate returns zero rows, or errors with `operator does not exist`. +- A write fails with `value for domain eql_v3_… violates check constraint`. +- Choosing the right operator for a column's domain. +- Ordering, ranging, or searching inside an encrypted JSON document. + +## The Two Halves + +Assume `users.email` is a `types.TextEq` column — the domain names below +follow the column, so a `types.TextSearch` column would use +`eql_v3.query_text_search` in exactly the same places. + +```ts +// 1. Encrypt for storage — the payload includes the ciphertext (`c`). +const enc = await client.encrypt('alice@example.com', { table: users, column: users.email }) +if (enc.failure) throw new Error(enc.failure.message) +await sql`INSERT INTO users (email) VALUES (${sql.json(enc.data)})` + +// 2. Mint a search needle — CIPHERTEXT-FREE, terms only. +const term = await client.encryptQuery('alice@example.com', { + table: users, column: users.email, queryType: 'equality', +}) +if (term.failure) throw new Error(term.failure.message) +const rows = await sql` + SELECT * FROM users WHERE email = ${sql.json(term.data)}::jsonb::eql_v3.query_text_eq` +``` + +Storage payloads and query terms are **different shapes with different +domains**. A storage payload carries `c` (the ciphertext); a query term +deliberately omits it and the `eql_v3.query_*` CHECKs *require* its absence. +Binding a storage payload where a query term belongs fails the CHECK, and vice +versa. + +`queryType` is one of `'equality'`, `'freeTextSearch'`, `'orderAndRange'`, +`'searchableJson'`. Omit it only for single-index columns (`types.TextEq`); +be explicit on multi-index domains like `types.TextSearch`. + +## Naming: Column Domain → Query Domain + +Strip `public.`, insert `query_`, move to the `eql_v3` schema: + +``` +public.eql_v3_text_eq → eql_v3.query_text_eq +public.eql_v3_text_search → eql_v3.query_text_search +public.eql_v3_bigint_ord → eql_v3.query_bigint_ord +public.eql_v3_timestamp_ord → eql_v3.query_timestamp_ord +``` + +**One irregular case:** `types.Json` builds `public.eql_v3_json_search`, but +its query domain is `eql_v3.query_json` — not `query_json_search`. + +| Schema factory | Column domain (`public.`) | Query domain (`eql_v3.`) | +|---|---|---| +| `types.TextEq` | `eql_v3_text_eq` | `query_text_eq` | +| `types.TextMatch` | `eql_v3_text_match` | `query_text_match` | +| `types.TextOrd` | `eql_v3_text_ord` | `query_text_ord` | +| `types.TextSearch` | `eql_v3_text_search` | `query_text_search` | +| `types.Eq` | `eql_v3__eq` | `query__eq` | +| `types.Ord` | `eql_v3__ord` | `query__ord` | +| `types.OrdOre` | `eql_v3__ord_ore` | `query__ord_ore` | +| `types.Json` | `eql_v3_json_search` | **`query_json`** | +| `types.Text`, `types.`, `types.Boolean` | `eql_v3_text` / `eql_v3_` / `eql_v3_boolean` | **none — storage only** | + +`` ranges over `Integer`, `Smallint`, `Bigint`, `Numeric`, `Real`, +`Double`, `Date`, `Timestamp`. The storage-only domains carry no query terms +by design — there is no query domain and nothing to search server-side. + +## The Predicate Matrix + +Which operators each column domain accepts against its query domain. Anything +not listed does not exist as an encrypted operator. + +| Column domain | Operators | Query domain operand | +|---|---|---| +| `eql_v3__eq`, `eql_v3_text_eq` | `=` `<>` | `query__eq` / `query_text_eq` | +| `eql_v3__ord`, `eql_v3_text_ord` | `=` `<>` `<` `<=` `>` `>=` | `query__ord` / `query_text_ord` | +| `eql_v3__ord_ore`, `eql_v3_text_ord_ore` | `=` `<>` `<` `<=` `>` `>=` | `query__ord_ore` / `query_text_ord_ore` | +| `eql_v3_text_match` | `@@` | `query_text_match` | +| `eql_v3_text_search` | `=` `<>` `<` `<=` `>` `>=` `@@` | `query_text_search` | +| `eql_v3_json_search` | `@>` | `query_json` | +| `eql_v3_json_entry` (from `col -> 'selector'`) | `=` `<>` `<` `<=` `>` `>=` | any `query__ord` / `query_text_ord` / `query_text_search` | + +Note what is **absent**: there is no `<` on an `_eq` domain, and no `@@` +outside the match-capable text domains. Asking for one raises +`operator does not exist` — which is the good failure. The bad failure is +leaving the operand as bare `jsonb` (see [Traps](#traps)). + +Every operator has a function twin, useful when an operator is awkward to +emit: `eql_v3.eq(col, term)`, `eql_v3.matches(col, term)`, and the comparison +functions. `col = term` and `eql_v3.eq(col, term)` are equivalent. + +## Binding Parameters: The Driver Rules + +**This differs between drivers.** Both encrypted payloads and query terms are +plain JS objects, and the two drivers disagree about how to put a JS object +into a `jsonb`-backed domain. + +### `postgres` (postgres-js) — always `sql.json(...)` + +| Binding form | `INSERT` into a domain column | Query operand with `::jsonb::eql_v3.query_*` | +|---|---|---| +| `${sql.json(payload)}` | ✅ | ✅ | +| `${payload}` (bare object) | ❌ `invalid input syntax for type json` | ✅ | +| `${JSON.stringify(payload)}::jsonb` | ❌ CHECK violation | ❌ CHECK violation | + +**Use `sql.json(...)` in both positions** — it is the only form that works in +both, so there is no reason to track which position you are in. + +```ts +await sql`INSERT INTO users (email) VALUES (${sql.json(enc.data)})` +await sql`SELECT * FROM users + WHERE email = ${sql.json(term.data)}::jsonb::eql_v3.query_text_eq` +``` + +### `pg` (node-postgres) — pass the object + +node-postgres serialises a JS object to JSON exactly once, so all three forms +happen to work. Pass the object and let the driver do it: + +```ts +await client.query('INSERT INTO users (email) VALUES ($1)', [enc.data]) +await client.query( + 'SELECT * FROM users WHERE email = $1::eql_v3.query_text_eq', [term.data]) +``` + +Do not pre-stringify even though `pg` tolerates it — it is the one habit that +silently breaks if the project ever moves to `postgres-js`. + +### The double-encoding failure, precisely + +`${JSON.stringify(payload)}::jsonb` on postgres-js produces: + +``` +value for domain eql_v3_text_search violates check constraint "eql_v3_text_search_check" +``` + +The message names neither JSON nor encoding, which is why this one costs an +afternoon. What happened: the explicit `::jsonb` makes postgres-js infer a +`jsonb` parameter, so it JSON-encodes the value — which was *already* a JSON +string. The result is a jsonb **string scalar**, not an object: + +```sql +SELECT jsonb_typeof($1::jsonb) -- 'string', not 'object' +``` + +Every EQL domain CHECK opens with `jsonb_typeof(VALUE) = 'object'`, so it +fails on the very first clause. Diagnose any CHECK-violation-on-write by +running `jsonb_typeof` on the parameter; `'string'` means double-encoded. + +## Query Recipes + +Assume `sql` is a postgres-js tag; for `pg` use numbered placeholders as above. + +### Equality + +```ts +const term = await client.encryptQuery(email, { + table: users, column: users.email, queryType: 'equality', +}) +await sql`SELECT * FROM users + WHERE email = ${sql.json(term.data)}::jsonb::eql_v3.query_text_eq` +``` + +On a `types.TextSearch` column the cast is `::eql_v3.query_text_search` — the +query domain always matches the *column's* domain, not the query type. + +### Free-text match + +```ts +// `bio` is a types.TextSearch column here; on a types.TextMatch column the +// cast is ::eql_v3.query_text_match. +const term = await client.encryptQuery('needle', { + table: users, column: users.bio, queryType: 'freeTextSearch', +}) +await sql`SELECT * FROM users + WHERE bio @@ ${sql.json(term.data)}::jsonb::eql_v3.query_text_search` +``` + +Match is **one-sided**: a hit may be a false positive, a miss never is. Filter +client-side after decryption if exactness matters — and never build a negated +match (`NOT (bio @@ …)`), which would drop true rows. Needles must be at least +3 characters; shorter ones tokenize to nothing and are rejected. + +### Range and ordering + +```ts +const term = await client.encryptQuery(new Date('2026-01-01'), { + table: events, column: events.createdAt, queryType: 'orderAndRange', +}) +await sql`SELECT * FROM events + WHERE created_at >= ${sql.json(term.data)}::jsonb::eql_v3.query_timestamp_ord + ORDER BY eql_v3.ord_term(created_at) DESC + LIMIT 20` +``` + +**`ORDER BY` must use the extractor form.** `ORDER BY created_at` sorts the +raw encrypted payload — which is neither meaningful nor index-backed. Sorting +on `eql_v3.ord_term(col)` is both. Ordering is available on `_ord`, +`_ord_ore`, and `text_search` columns; use `ord_term_ore` for `_ord_ore`. + +### Encrypted JSON — containment + +```ts +const needle = await client.encryptQuery({ role: 'admin' }, { + table: users, column: users.prefs, queryType: 'searchableJson', +}) +await sql`SELECT * FROM users + WHERE prefs @> ${sql.json(needle.data)}::jsonb::eql_v3.query_json` +``` + +An **object** value produces a containment needle. Note the containment needle +is a bare `{ sv: [...] }` shape with no version field — unlike the scalar +terms, which are full v3 envelopes. Bind it the same way regardless. + +### Encrypted JSON — field selector + +A **string** value produces a JSONPath selector, and v3 has no encrypted-selector +envelope: `encryptQuery` returns the **bare selector-hash string**. Bind it as +the plain text argument of `->` / `->>`, with no domain cast: + +```ts +const sel = await client.encryptQuery('$.role', { + table: users, column: users.prefs, queryType: 'searchableJson', +}) +await sql`SELECT prefs -> ${sel.data} FROM users` +``` + +The extracted value is an `eql_v3_json_entry`, which accepts the ordering +operators — so a field inside an encrypted document can be ranged and ordered: + +```ts +await sql`SELECT * FROM orders + WHERE data -> ${sel.data} >= ${sql.json(term.data)}::jsonb::eql_v3.query_integer_ord + ORDER BY eql_v3.ord_term(data -> ${sel.data})` +``` + +Field-level `=` between extracted entries is **not** supported (an extracted +entry carries no value selector) — use document containment for exact field +equality. + +## Reading Rows Back + +`SELECT` returns the stored payload as an object; hand it straight to +`decrypt` — do not `JSON.parse` it, and do not cast it to `::jsonb` in the +query (see the projection trap below). + +```ts +const [row] = await sql`SELECT id, email FROM users WHERE id = ${id}` +const dec = await client.decrypt(row.email) +if (dec.failure) throw new Error(dec.failure.message) +``` + +For whole rows, `decryptModel` / `bulkDecryptModels` walk the schema and +decrypt every declared column in one ZeroKMS round trip. They match by **JS +property name**, so a raw `SELECT` returning snake_case DB column names will +not match a schema keyed by camelCase properties — alias in the query +(`SELECT last_login AS "lastLogin"`) or decrypt the columns individually. + +## Traps + +**A bare `::jsonb` operand picks a different operator, not a missing one.** +EQL also defines overloads with `jsonb` on the right — and they coerce that +operand to the **storage** domain: + +```sql +-- what `col = $1::jsonb` actually resolves to: +eql_v3.eq_term(a) = eql_v3.eq_term(b::public.eql_v3_text_search) +``` + +The storage domain's CHECK requires the ciphertext key `c`, which query terms +deliberately omit — so binding a query term without the domain cast raises a +CHECK violation rather than doing what you meant. Those overloads exist so you +can compare against a *full storage envelope* (an already-encrypted value). +For a needle from `encryptQuery`, always cast to the `eql_v3.query_*` domain. + +**Cast to the `query_*` domain, not the column domain.** `$1::public.eql_v3_text_eq` +fails for the same reason — the column domain's CHECK requires `c`. + +**The `value::jsonb` projection trap.** `SELECT email::jsonb … ORDER BY email` +folds the cast into the scan and sorts on `(email)::jsonb`, matching no index. +Project the column raw. + +**`GROUP BY` / `DISTINCT` on the raw column** hashes the whole encrypted +payload (1–2 KB per row) and spills. Group on the extractor — +`GROUP BY eql_v3.eq_term(email)` — which is small and deterministic. + +**Predicates are not indexes.** Everything here works without an index and +sequential-scans. Adding the functional index over the extractor is a separate +step — see `stash-indexing`. + +**Every writer needs the same credentials.** Index terms derive from the +ZeroKMS client key, so rows written by a client with different `CS_*` +credentials decrypt correctly but never match a query — silently. This +includes `stash encrypt backfill` and seed scripts. See `stash-edge` § +The Credential-Identity Rule. + +## Troubleshooting + +**`operator does not exist: public.eql_v3_… = eql_v3.query_…`** — the domain +pair has no such operator. Check the [matrix](#the-predicate-matrix): the +column's domain may not support that predicate (e.g. `<` on an `_eq` column), +or the query domain does not match the column's domain. + +**`value for domain … violates check constraint`** on write — double-encoded +payload; run `SELECT jsonb_typeof($1::jsonb)` and see +[above](#the-double-encoding-failure-precisely). On a *query* operand, the +same error usually means a storage payload (with `c`) was bound where a query +term belongs. + +**Zero rows, no error** — in order of likelihood: (1) the rows were written +under different `CS_*` credentials (see the trap above — this is the common +one, and it is completely silent); (2) the column's domain does not carry the +term the predicate needs (a `types.Text` column carries none); (3) a +free-text needle under 3 characters tokenized to nothing. A *missing* domain +cast raises an error rather than returning zero rows, so it is not a candidate +here. + +**Slow but correct** — no index. See `stash-indexing`; confirm with +`EXPLAIN (COSTS OFF)` that the plan shows an `Index Cond` on the extractor +rather than a `Seq Scan`. + +**Checking what a column actually is**, when the schema and the database may +have drifted: + +```sql +SELECT column_name, domain_schema, domain_name + FROM information_schema.columns + WHERE table_name = 'users' AND domain_name LIKE 'eql_v3%'; +``` + +## Reference + +- `stash-encryption` — schema authoring, the `types.*` catalog, `encryptQuery` + and the client API, the rollout/cutover lifecycle. +- `stash-indexing` — functional indexes over the term extractors, and the + `EXPLAIN` checklist. +- `stash-edge` — the WASM entry, `CS_*` credentials, and the + credential-identity rule. +- `stash-cli` — `stash eql install`, `stash db validate`, `stash encrypt backfill`. diff --git a/skills/stash-supabase/SKILL.md b/skills/stash-supabase/SKILL.md index 2cb45f9a..16e04d81 100644 --- a/skills/stash-supabase/SKILL.md +++ b/skills/stash-supabase/SKILL.md @@ -56,6 +56,14 @@ this is also how **Supabase Edge Functions** get credentials in local dev — `stash env --name edge-dev --write` and pass `--env-file`, or `supabase secrets set` them for deploys. +> **One credential per environment, used by everything that writes.** EQL index +> terms derive from the ZeroKMS client key, so rows written by a client with +> different `CS_*` values decrypt correctly but never match a query — silently. +> That includes `stash encrypt backfill`, seed scripts, and Edge Functions. +> Encryption *inside* an Edge Function (Deno, no native modules) uses the +> `@cipherstash/stack/wasm-inline` entry — see the `stash-edge` skill; SQL +> written by hand in a migration or RPC is covered by `stash-sql`. + ### 1. Install EQL v3 on the database ```bash From cb4b50a28b828a0c0a791f0ec908fb6401801427 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 25 Jul 2026 18:19:57 +1000 Subject: [PATCH 2/9] refactor(skills): rename stash-sql to stash-postgres MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The skill is Postgres-specific throughout — encrypted columns are Postgres domains over jsonb, the driver rules cover pg and postgres-js, and the drift check reads information_schema. Nothing in it generalises to another SQL dialect, so the name overclaimed. Renames the directory and frontmatter name, updates the SKILL_MAP entries in both the CLI and wizard installers, the setup-prompt purpose line, the cross-links in the five sibling skills, and the AGENTS.md skill list and change→skill map. The skill has not shipped yet (no CHANGELOG entry — only the pending changeset), so no installed .claude/skills/ directory carries the old name and the existing changeset is amended in place rather than adding a second one. The new name also matches the CLI's `postgresql` integration key, which is the no-ORM path that installs it. --- ...h-sql-edge-skills.md => stash-postgres-edge-skills.md} | 4 ++-- AGENTS.md | 4 ++-- .../commands/init/lib/__tests__/install-skills.test.ts | 4 ++-- packages/cli/src/commands/init/lib/install-skills.ts | 6 +++--- packages/cli/src/commands/init/lib/setup-prompt.ts | 4 ++-- packages/wizard/src/lib/install-skills.ts | 6 +++--- skills/stash-drizzle/SKILL.md | 2 +- skills/stash-edge/SKILL.md | 8 ++++---- skills/stash-encryption/SKILL.md | 2 +- skills/stash-indexing/SKILL.md | 4 ++-- skills/{stash-sql => stash-postgres}/SKILL.md | 6 +++--- skills/stash-prisma-next/SKILL.md | 2 +- skills/stash-supabase/SKILL.md | 2 +- 13 files changed, 27 insertions(+), 27 deletions(-) rename .changeset/{stash-sql-edge-skills.md => stash-postgres-edge-skills.md} (95%) rename skills/{stash-sql => stash-postgres}/SKILL.md (95%) diff --git a/.changeset/stash-sql-edge-skills.md b/.changeset/stash-postgres-edge-skills.md similarity index 95% rename from .changeset/stash-sql-edge-skills.md rename to .changeset/stash-postgres-edge-skills.md index 1b6d8ca7..825c2f76 100644 --- a/.changeset/stash-sql-edge-skills.md +++ b/.changeset/stash-postgres-edge-skills.md @@ -5,14 +5,14 @@ --- Two new bundled agent skills for the integrations that don't use an ORM — -`stash-sql` and `stash-edge` (#754). +`stash-postgres` and `stash-edge` (#754). Everything a raw-SQL or edge integration needed was reachable only from `dist/*.d.ts` JSDoc, the Postgres catalog, or experiment: grepping the skills `stash init` installs for `postgres-js|::jsonb::eql|sql.json|query_text_search` returned a single hit, in an unrelated code comment. -**`stash-sql`** — hand-written SQL over `pg` / `postgres-js`, no ORM. The +**`stash-postgres`** — hand-written SQL over `pg` / `postgres-js`, no ORM. The column-domain-to-query-domain operator matrix (which of `=`, `<>`, `<`, `>=`, `@@`, `@>` each encrypted domain accepts, and against which `eql_v3.query_*` operand), the storage-vs-query payload distinction, per-driver parameter diff --git a/AGENTS.md b/AGENTS.md index bd5c17b0..2ce2f337 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -85,7 +85,7 @@ If these variables are missing, tests that require live encryption will fail or - `e2e/*`: Cross-package end-to-end tests (package managers, supply chain, Prisma example README) - `examples/*`: Working apps (basic, prisma, supabase-worker) - `docs/plans/*`: Internal design plans. User-facing documentation lives at https://cipherstash.com/docs (not in this repo). -- `skills/*`: Agent skills (`stash-cli`, `stash-encryption`, `stash-indexing`, `stash-sql`, `stash-edge`, `stash-drizzle`, `stash-dynamodb`, `stash-supabase`, `stash-prisma-next`, `stash-supply-chain-security`) +- `skills/*`: Agent skills (`stash-cli`, `stash-encryption`, `stash-indexing`, `stash-postgres`, `stash-edge`, `stash-drizzle`, `stash-dynamodb`, `stash-supabase`, `stash-prisma-next`, `stash-supply-chain-security`) ## Agent Skills — these ship to customers @@ -113,7 +113,7 @@ nothing type-checks them, and the damage lands in a customer's repo, not ours. | Drizzle / Supabase / Prisma Next / DynamoDB integrations | `skills/stash-drizzle`, `skills/stash-supabase`, `skills/stash-prisma-next`, `skills/stash-dynamodb` | | The rollout/cutover lifecycle (`packages/migrate`, `stash encrypt *`) | `skills/stash-encryption` and `skills/stash-cli` | | The `@cipherstash/eql` pin, `eql install`/`eql migration` behaviour, or index-related SQL guidance | `skills/stash-indexing` | -| The EQL operator/domain surface (`eql_v3.query_*` casts, predicate forms) | `skills/stash-sql` | +| The EQL operator/domain surface (`eql_v3.query_*` casts, predicate forms) | `skills/stash-postgres` | | `packages/stack/src/wasm-inline.ts`, the WASM entry's exports, or `stash env` | `skills/stash-edge` | | pnpm config, CI workflows, dependency policy | `skills/stash-supply-chain-security` | | The durable agent rules themselves | `packages/cli/src/commands/init/doctrine/AGENTS-doctrine.md` | diff --git a/packages/cli/src/commands/init/lib/__tests__/install-skills.test.ts b/packages/cli/src/commands/init/lib/__tests__/install-skills.test.ts index bc9d6ec4..052496cc 100644 --- a/packages/cli/src/commands/init/lib/__tests__/install-skills.test.ts +++ b/packages/cli/src/commands/init/lib/__tests__/install-skills.test.ts @@ -92,7 +92,7 @@ describe('SKILL_MAP', () => { 'postgresql', 'supabase', ] as const)('%s includes the raw-SQL and edge skills', (integration) => { - expect(SKILL_MAP[integration]).toContain('stash-sql') + expect(SKILL_MAP[integration]).toContain('stash-postgres') expect(SKILL_MAP[integration]).toContain('stash-edge') }) @@ -102,7 +102,7 @@ describe('SKILL_MAP', () => { 'drizzle', 'prisma-next', ] as const)('%s does not install the raw-SQL skill', (integration) => { - expect(SKILL_MAP[integration]).not.toContain('stash-sql') + expect(SKILL_MAP[integration]).not.toContain('stash-postgres') }) }) diff --git a/packages/cli/src/commands/init/lib/install-skills.ts b/packages/cli/src/commands/init/lib/install-skills.ts index a57897d4..8353ce1b 100644 --- a/packages/cli/src/commands/init/lib/install-skills.ts +++ b/packages/cli/src/commands/init/lib/install-skills.ts @@ -20,7 +20,7 @@ export const SKILL_MAP: Record = { 'stash-encryption', 'stash-supabase', 'stash-indexing', - 'stash-sql', + 'stash-postgres', 'stash-edge', 'stash-cli', ], @@ -30,13 +30,13 @@ export const SKILL_MAP: Record = { 'stash-indexing', 'stash-cli', ], - // The no-ORM path: `stash-sql` (binding + predicate forms) and `stash-edge` + // The no-ORM path: `stash-postgres` (binding + predicate forms) and `stash-edge` // (WASM entry, CS_* credentials) are the two skills this integration has no // other source for — everything else assumes an ORM emits the operands. postgresql: [ 'stash-encryption', 'stash-indexing', - 'stash-sql', + 'stash-postgres', 'stash-edge', 'stash-cli', ], diff --git a/packages/cli/src/commands/init/lib/setup-prompt.ts b/packages/cli/src/commands/init/lib/setup-prompt.ts index b3aefdbb..b09f1ecb 100644 --- a/packages/cli/src/commands/init/lib/setup-prompt.ts +++ b/packages/cli/src/commands/init/lib/setup-prompt.ts @@ -110,8 +110,8 @@ const SKILL_PURPOSES: Record = { 'Prisma Next-specific patterns: `cipherstash.*` field constructors, migration flow, encrypted query operators', 'stash-indexing': 'index recipes for encrypted columns — the `eql_v3` extractor functional indexes, Supabase/managed-Postgres constraints, EXPLAIN verification', - 'stash-sql': - 'hand-written SQL over `pg` / `postgres-js`: the encrypted predicate matrix, `eql_v3.query_*` operand casts, per-driver parameter binding', + 'stash-postgres': + 'hand-written Postgres SQL over `pg` / `postgres-js`: the encrypted predicate matrix, `eql_v3.query_*` operand casts, per-driver parameter binding', 'stash-edge': 'the `@cipherstash/stack/wasm-inline` entry for Deno / Supabase Edge Functions / Workers: imports, `CS_*` credentials, the credential-identity rule', 'stash-dynamodb': diff --git a/packages/wizard/src/lib/install-skills.ts b/packages/wizard/src/lib/install-skills.ts index 25442278..b3f5d2f9 100644 --- a/packages/wizard/src/lib/install-skills.ts +++ b/packages/wizard/src/lib/install-skills.ts @@ -13,13 +13,13 @@ import type { Integration } from './types.js' */ const SKILL_MAP: Record = { drizzle: ['stash-encryption', 'stash-drizzle', 'stash-indexing', 'stash-cli'], - // `stash-sql` / `stash-edge` mirror the CLI's SKILL_MAP — see the comments + // `stash-postgres` / `stash-edge` mirror the CLI's SKILL_MAP — see the comments // there for why Supabase and the generic (no-ORM) path get them (#754). supabase: [ 'stash-encryption', 'stash-supabase', 'stash-indexing', - 'stash-sql', + 'stash-postgres', 'stash-edge', 'stash-cli', ], @@ -27,7 +27,7 @@ const SKILL_MAP: Record = { generic: [ 'stash-encryption', 'stash-indexing', - 'stash-sql', + 'stash-postgres', 'stash-edge', 'stash-cli', ], diff --git a/skills/stash-drizzle/SKILL.md b/skills/stash-drizzle/SKILL.md index 8a4286d2..bc743651 100644 --- a/skills/stash-drizzle/SKILL.md +++ b/skills/stash-drizzle/SKILL.md @@ -417,7 +417,7 @@ import { index } from "drizzle-orm/pg-core" Run `ANALYZE
` after the migration applies — an expression index gathers no statistics at `CREATE INDEX` time. For when to create indexes during a rollout (after backfill, before switching reads), engagement rules, and `EXPLAIN` verification, see the `stash-indexing` skill. -> **Dropping to raw SQL?** `db.execute(sql\`…\`)` bypasses the operators this integration emits, so you own the operand casts yourself — an encrypted predicate needs its needle cast to the column's `eql_v3.query_*` domain, and the driver's parameter-binding rules differ between `pg` and `postgres-js`. The `stash-sql` skill is the reference for both. +> **Dropping to raw SQL?** `db.execute(sql\`…\`)` bypasses the operators this integration emits, so you own the operand casts yourself — an encrypted predicate needs its needle cast to the column's `eql_v3.query_*` domain, and the driver's parameter-binding rules differ between `pg` and `postgres-js`. The `stash-postgres` skill is the reference for both. ## Migrating an Existing Column to Encrypted diff --git a/skills/stash-edge/SKILL.md b/skills/stash-edge/SKILL.md index 2359d39f..0ea023cc 100644 --- a/skills/stash-edge/SKILL.md +++ b/skills/stash-edge/SKILL.md @@ -14,7 +14,7 @@ and nothing for a bundler to externalise. This skill covers that entry and the deployment shape around it. It is EQL v3 throughout. For the SQL that actually queries the encrypted columns — the -predicate forms and driver binding rules — see `stash-sql`; edge functions +predicate forms and driver binding rules — see `stash-postgres`; edge functions almost always talk to Postgres over a raw driver, so the two are usually read together. @@ -352,7 +352,7 @@ const rows = await sql` ``` The predicate forms, the per-driver binding rules, and the query-domain names -are the subject of `stash-sql` — read it before writing the first query. The +are the subject of `stash-postgres` — read it before writing the first query. The two rules that bite immediately: the operand must be **cast to the column's `eql_v3.query_*` domain**, and on `postgres-js` payloads must be bound with `sql.json(...)`, never pre-stringified. @@ -375,7 +375,7 @@ opaquely; the example in `examples/supabase-worker` does exactly this. **Encryption works, search returns zero rows** — the credential-identity rule above. Second most likely: a missing index (see `stash-indexing`) makes it slow, not empty, so empty results point at credentials or an untyped operand -(`stash-sql`). +(`stash-postgres`). **Search needle rejected** — free-text needles must be at least 3 characters; shorter ones tokenize to nothing. @@ -386,7 +386,7 @@ user-scoped, so it is reused across invocations on a warm isolate. ## Reference -- `stash-sql` — the raw-SQL predicate cookbook and driver binding rules. +- `stash-postgres` — the raw-SQL predicate cookbook and driver binding rules. - `stash-encryption` — schema authoring, the `types.*` domain catalog, and the rollout/cutover lifecycle. - `stash-cli` — `stash env`, `stash eql install`, `stash encrypt backfill`. diff --git a/skills/stash-encryption/SKILL.md b/skills/stash-encryption/SKILL.md index 02e0a78e..50af6578 100644 --- a/skills/stash-encryption/SKILL.md +++ b/skills/stash-encryption/SKILL.md @@ -570,7 +570,7 @@ All values in the array must be non-null. ### On the Wire: Operators and Ordering -Scalar filters compare through each domain's `eql_v3.*` operators (`col = term`, `col > term`, ...), and `ORDER BY` on an encrypted column goes through the ordering extractors — `eql_v3.ord_term(col)` for OPE-backed (`Ord`/`Search`) domains, `eql_v3.ord_term_ore(col)` for `OrdOre`. The Drizzle v3 integration emits all of this for you (including `asc`/`desc`, which emit `ORDER BY eql_v3.ord_term(col)`). Over Supabase/PostgREST, the adapter's `order()` works on OPE-backed ordering columns (plain `*_ord`, `text_ord`, `text_search`) by sorting on the column's `col->op` term; `OrdOre`-flavour (`*_ord_ore`) domains and columns with no ordering term are rejected. See the `stash-drizzle` and `stash-supabase` skills. These same extractor expressions are also what you index — the functional-index recipes are in the `stash-indexing` skill. Writing the SQL by hand instead (no ORM, `pg` / `postgres-js`)? The predicate matrix, the `eql_v3.query_*` operand casts, and the per-driver parameter-binding rules are in the `stash-sql` skill; running on Deno / Workers / Supabase Edge Functions, the `stash-edge` skill. +Scalar filters compare through each domain's `eql_v3.*` operators (`col = term`, `col > term`, ...), and `ORDER BY` on an encrypted column goes through the ordering extractors — `eql_v3.ord_term(col)` for OPE-backed (`Ord`/`Search`) domains, `eql_v3.ord_term_ore(col)` for `OrdOre`. The Drizzle v3 integration emits all of this for you (including `asc`/`desc`, which emit `ORDER BY eql_v3.ord_term(col)`). Over Supabase/PostgREST, the adapter's `order()` works on OPE-backed ordering columns (plain `*_ord`, `text_ord`, `text_search`) by sorting on the column's `col->op` term; `OrdOre`-flavour (`*_ord_ore`) domains and columns with no ordering term are rejected. See the `stash-drizzle` and `stash-supabase` skills. These same extractor expressions are also what you index — the functional-index recipes are in the `stash-indexing` skill. Writing the SQL by hand instead (no ORM, `pg` / `postgres-js`)? The predicate matrix, the `eql_v3.query_*` operand casts, and the per-driver parameter-binding rules are in the `stash-postgres` skill; running on Deno / Workers / Supabase Edge Functions, the `stash-edge` skill. ## Encrypted JSON (`types.Json`) diff --git a/skills/stash-indexing/SKILL.md b/skills/stash-indexing/SKILL.md index 8ed63bf3..d7f7d981 100644 --- a/skills/stash-indexing/SKILL.md +++ b/skills/stash-indexing/SKILL.md @@ -257,7 +257,7 @@ Index not being used: - **Drizzle** — `encryptedIndexes(t)` from `@cipherstash/stack-drizzle` derives the recommended indexes for every encrypted column in the table, or declare individual expression indexes in the schema DSL. See `stash-drizzle` § Indexing Encrypted Columns. - **Prisma Next** — Prisma's schema language cannot express functional indexes; the DDL goes in a migration in the adapter's flow. See `stash-prisma-next`. - **Supabase** — a `supabase/migrations/` file; no superuser needed (see above). See `stash-supabase`. -- **Raw SQL / plain PostgreSQL** — the recipes in this skill, in whatever migration tool owns the schema. Never ad-hoc in production. The predicates those indexes serve are in `stash-sql`. +- **Raw SQL / plain PostgreSQL** — the recipes in this skill, in whatever migration tool owns the schema. Never ad-hoc in production. The predicates those indexes serve are in `stash-postgres`. ## When to Create Indexes During an Encryption Rollout @@ -271,5 +271,5 @@ Index not being used: - `stash-encryption` — the `types.*` domain catalog, wire-format operators and ordering, and the rollout/cutover lifecycle. - `stash-cli` — `stash eql install`, `stash db validate` (its "No indexes on an encrypted column" Info finding is resolved by this skill), `stash encrypt backfill` / `cutover`. - `stash-drizzle`, `stash-supabase`, `stash-prisma-next` — per-integration query patterns; index DDL placement per the section above. -- `stash-sql` — the hand-written predicate forms these indexes serve (`pg` / `postgres-js`, no ORM). +- `stash-postgres` — the hand-written predicate forms these indexes serve (`pg` / `postgres-js`, no ORM). - `stash-edge` — the WASM entry, for apps whose queries run on Deno / Workers / Supabase Edge Functions. diff --git a/skills/stash-sql/SKILL.md b/skills/stash-postgres/SKILL.md similarity index 95% rename from skills/stash-sql/SKILL.md rename to skills/stash-postgres/SKILL.md index 3846b3d5..6f07c05f 100644 --- a/skills/stash-sql/SKILL.md +++ b/skills/stash-postgres/SKILL.md @@ -1,9 +1,9 @@ --- -name: stash-sql -description: Query EQL v3 encrypted columns from hand-written SQL over `pg` (node-postgres) or `postgres` (postgres-js) — no ORM. Covers the column-domain-to-query-domain operator matrix (which of `=`, `<>`, `<`, `>=`, `@@`, `@>` each encrypted domain accepts), minting search needles with `encryptQuery`, the per-driver parameter-binding rules for encrypted payloads, and the double-encoding failure that trips the domain CHECK with a message naming neither JSON nor encoding. Use when writing INSERT/SELECT against an encrypted column without an ORM, when a predicate returns zero rows or raises "operator does not exist", or when a domain CHECK constraint rejects an encrypted value on write. +name: stash-postgres +description: Query EQL v3 encrypted columns from hand-written Postgres SQL over `pg` (node-postgres) or `postgres` (postgres-js) — no ORM. Covers the column-domain-to-query-domain operator matrix (which of `=`, `<>`, `<`, `>=`, `@@`, `@>` each encrypted domain accepts), minting search needles with `encryptQuery`, the per-driver parameter-binding rules for encrypted payloads, and the double-encoding failure that trips the domain CHECK with a message naming neither JSON nor encoding. Use when writing INSERT/SELECT against an encrypted column without an ORM, when a predicate returns zero rows or raises "operator does not exist", or when a domain CHECK constraint rejects an encrypted value on write. --- -# Raw SQL Against Encrypted Columns (EQL v3) +# Raw Postgres SQL Against Encrypted Columns (EQL v3) An EQL v3 encrypted column is a **Postgres domain over `jsonb`** (`public.eql_v3_text_search`, `public.eql_v3_bigint_ord`, …). Reading and writing it from raw SQL is two rules: diff --git a/skills/stash-prisma-next/SKILL.md b/skills/stash-prisma-next/SKILL.md index 11e8b448..fef86987 100644 --- a/skills/stash-prisma-next/SKILL.md +++ b/skills/stash-prisma-next/SKILL.md @@ -181,7 +181,7 @@ model — which domains take which index, engagement rules, `EXPLAIN` verification, rollout timing — see the `stash-indexing` skill. For encrypted predicates written as raw SQL rather than through the `cipherstash:*` operators — operand casts to `eql_v3.query_*`, per-driver parameter binding — -see the `stash-sql` skill. +see the `stash-postgres` skill. In a migration, the recipes ride a raw-SQL operation (`rawSql` from `@prisma-next/postgres/migration`) in the migration's `operations`: diff --git a/skills/stash-supabase/SKILL.md b/skills/stash-supabase/SKILL.md index 16e04d81..b1d6e6b1 100644 --- a/skills/stash-supabase/SKILL.md +++ b/skills/stash-supabase/SKILL.md @@ -62,7 +62,7 @@ this is also how **Supabase Edge Functions** get credentials in local dev — > That includes `stash encrypt backfill`, seed scripts, and Edge Functions. > Encryption *inside* an Edge Function (Deno, no native modules) uses the > `@cipherstash/stack/wasm-inline` entry — see the `stash-edge` skill; SQL -> written by hand in a migration or RPC is covered by `stash-sql`. +> written by hand in a migration or RPC is covered by `stash-postgres`. ### 1. Install EQL v3 on the database From 38e99298254850acd68f74f8aa5d3b26f7855816 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 25 Jul 2026 18:25:14 +1000 Subject: [PATCH 3/9] docs(skills): scope stash-postgres against Proxy and cite EQL upstream MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The skill covered client-side encryption over a direct connection without ever saying so. That is precisely the fork `stash init` asks about and stores as `usesProxy`: a CipherStash Proxy user writes plaintext SQL and Proxy encrypts on the wire, so every rule in the skill — mint a term with encryptQuery, cast to eql_v3.query_*, bind with sql.json — is wrong for them, and nothing said which world they were in. Adds a scope callout up front, and notes that Proxy's config lifecycle (stash db push into eql_v2_configuration) is the EQL v2 one, while this skill is v3 where there is nothing to push. The operator matrix also read as if the client library defined it. It does not: the domains, operators, CHECKs, and extractors all come from the EQL bundle developed at cipherstash/encrypt-query-language and shipped as @cipherstash/eql. A reader hitting an operator the matrix does not list had nowhere authoritative to check and no idea where to file it. Adds a provenance section with the version check (SELECT eql_v3.version()), ties the "operator does not exist" troubleshooting entry to it, and adds upstream links for EQL and Proxy to the Reference list. Also fixes the frontmatter description, which now states the direct-connection assumption so skill selection reflects it. --- .changeset/stash-postgres-edge-skills.md | 7 ++- skills/stash-postgres/SKILL.md | 63 ++++++++++++++++++++++-- 2 files changed, 66 insertions(+), 4 deletions(-) diff --git a/.changeset/stash-postgres-edge-skills.md b/.changeset/stash-postgres-edge-skills.md index 825c2f76..9375e148 100644 --- a/.changeset/stash-postgres-edge-skills.md +++ b/.changeset/stash-postgres-edge-skills.md @@ -23,7 +23,12 @@ on postgres-js double-encodes it into a jsonb *string* scalar, tripping the domain CHECK with a message naming neither JSON nor encoding; and leaving an operand as bare `jsonb` silently selects a different operator overload — one that coerces to the *storage* domain and so rejects the ciphertext-free query -term. +term. It also scopes itself against the two things a hand-written-SQL reader +is otherwise left to infer: **CipherStash Proxy** (where you write plaintext +SQL and none of the skill applies — the `usesProxy` fork `stash init` already +asked about), and the provenance of the operator surface itself (the EQL +bundle from `cipherstash/encrypt-query-language`, version-checkable with +`SELECT eql_v3.version()`, and where operator gaps should be filed). **`stash-edge`** — the `@cipherstash/stack/wasm-inline` entry for Deno, Supabase Edge Functions, Cloudflare Workers, and Bun. Import specifier per diff --git a/skills/stash-postgres/SKILL.md b/skills/stash-postgres/SKILL.md index 6f07c05f..25eaa60b 100644 --- a/skills/stash-postgres/SKILL.md +++ b/skills/stash-postgres/SKILL.md @@ -1,6 +1,6 @@ --- name: stash-postgres -description: Query EQL v3 encrypted columns from hand-written Postgres SQL over `pg` (node-postgres) or `postgres` (postgres-js) — no ORM. Covers the column-domain-to-query-domain operator matrix (which of `=`, `<>`, `<`, `>=`, `@@`, `@>` each encrypted domain accepts), minting search needles with `encryptQuery`, the per-driver parameter-binding rules for encrypted payloads, and the double-encoding failure that trips the domain CHECK with a message naming neither JSON nor encoding. Use when writing INSERT/SELECT against an encrypted column without an ORM, when a predicate returns zero rows or raises "operator does not exist", or when a domain CHECK constraint rejects an encrypted value on write. +description: Query EQL v3 encrypted columns from hand-written Postgres SQL over `pg` (node-postgres) or `postgres` (postgres-js) — no ORM. Covers the column-domain-to-query-domain operator matrix (which of `=`, `<>`, `<`, `>=`, `@@`, `@>` each encrypted domain accepts), minting search needles with `encryptQuery`, the per-driver parameter-binding rules for encrypted payloads, and the double-encoding failure that trips the domain CHECK with a message naming neither JSON nor encoding. Use when writing INSERT/SELECT against an encrypted column without an ORM, when a predicate returns zero rows or raises "operator does not exist", or when a domain CHECK constraint rejects an encrypted value on write. Assumes a direct Postgres connection with client-side encryption — CipherStash Proxy encrypts on the wire and needs none of this. --- # Raw Postgres SQL Against Encrypted Columns (EQL v3) @@ -22,6 +22,23 @@ services, Hono, edge functions. If you use Drizzle, Prisma Next, or the Supabase client, those integrations emit correct operands for you: see `stash-drizzle`, `stash-prisma-next`, `stash-supabase` instead. +> **Using CipherStash Proxy? None of this applies.** This skill assumes the +> app connects to Postgres **directly** and encrypts **client-side**: Stack +> mints the payloads and the query terms, and your SQL carries them. Through +> [CipherStash Proxy](https://github.com/cipherstash/proxy) the split is the +> opposite — you write *plaintext* SQL and Proxy encrypts on write and +> decrypts on read, so there is no `encryptQuery`, no `eql_v3.query_*` cast, +> and no payload to bind. `stash init` already asked which path you are on +> (`--proxy` / `--no-proxy`) and recorded the answer as `usesProxy` in +> `.cipherstash/context.json` — read it there if you are unsure. +> +> Proxy also has its own schema lifecycle — `stash db push` into +> `eql_v2_configuration`, promoted at cutover — and that lifecycle is the EQL +> **v2** one. This skill is EQL v3, where a column's encryption config lives +> in its domain and there is no configuration table to push (`stash db push` +> against a v3 database says exactly that and refuses). See `stash-encryption` +> § If you use CipherStash Proxy for the Proxy-side rollout. + ## When to Use This Skill - Writing `INSERT` / `UPDATE` / `SELECT` against an encrypted column by hand. @@ -115,6 +132,30 @@ Every operator has a function twin, useful when an operator is awkward to emit: `eql_v3.eq(col, term)`, `eql_v3.matches(col, term)`, and the comparison functions. `col = term` and `eql_v3.eq(col, term)` are equivalent. +### Where this surface is defined + +Nothing in the matrix above is authored by the client library. The domains, +operators, CHECKs, and extractor functions all come from the **EQL SQL +bundle** that `stash eql install` applies — published as the `@cipherstash/eql` +package and developed at +[`cipherstash/encrypt-query-language`](https://github.com/cipherstash/encrypt-query-language). +The CLI pins an exact version, so a database is only ever on one bundle. + +That makes the matrix a snapshot of a *versioned* surface. Check it against +the bundle actually installed before concluding an operator is missing: + +```sql +SELECT eql_v3.version(); -- e.g. '3.0.2' +``` + +If an operator is absent from both this matrix and your installed bundle, +that is an upstream question, not a client-library one — the operator set +lives in `encrypt-query-language`. The bundle is also readable locally — the +`stash` CLI depends on `@cipherstash/eql`, which ships the install SQL at +`dist/sql/cipherstash-encrypt.sql` (under `node_modules`, wherever your +package manager resolves it). That file is the last word on which overloads +exist. + ## Binding Parameters: The Driver Rules **This differs between drivers.** Both encrypted payloads and query terms are @@ -325,7 +366,10 @@ The Credential-Identity Rule. **`operator does not exist: public.eql_v3_… = eql_v3.query_…`** — the domain pair has no such operator. Check the [matrix](#the-predicate-matrix): the column's domain may not support that predicate (e.g. `<` on an `_eq` column), -or the query domain does not match the column's domain. +or the query domain does not match the column's domain. If the matrix says +the operator *should* exist, check the installed bundle version +(`SELECT eql_v3.version()`) — an older EQL install can predate an overload +documented here. See [Where this surface is defined](#where-this-surface-is-defined). **`value for domain … violates check constraint`** on write — double-encoded payload; run `SELECT jsonb_typeof($1::jsonb)` and see @@ -349,6 +393,8 @@ rather than a `Seq Scan`. have drifted: ```sql +SELECT eql_v3.version(); -- which bundle defines the operators available + SELECT column_name, domain_schema, domain_name FROM information_schema.columns WHERE table_name = 'users' AND domain_name LIKE 'eql_v3%'; @@ -362,4 +408,15 @@ SELECT column_name, domain_schema, domain_name `EXPLAIN` checklist. - `stash-edge` — the WASM entry, `CS_*` credentials, and the credential-identity rule. -- `stash-cli` — `stash eql install`, `stash db validate`, `stash encrypt backfill`. +- `stash-cli` — `stash eql install`, `stash db validate`, `stash encrypt backfill`, + and the `--proxy` / `--no-proxy` choice behind `usesProxy`. + +Upstream: + +- [`cipherstash/encrypt-query-language`](https://github.com/cipherstash/encrypt-query-language) + — EQL itself: the authoritative definition of every domain, operator, CHECK, + and extractor named in this skill, shipped as `@cipherstash/eql`. Operator + gaps and domain-level bugs belong there, not against the client library. +- [`cipherstash/proxy`](https://github.com/cipherstash/proxy) — CipherStash + Proxy, the alternative to this entire skill: plaintext SQL, encryption on + the wire. From beedea2e8fb484e328d1cb6026dfa9d22f9a242b Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 25 Jul 2026 19:08:56 +1000 Subject: [PATCH 4/9] docs(skills): drop the usesProxy pointer from the stash-postgres Proxy callout MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The callout told readers to check `usesProxy` in `.cipherstash/context.json` to learn which path they were on. That flag and field are being removed (the CLI never used the value for anything beyond gating a wizard `stash db push` step), so the pointer would have shipped stale. The Proxy scoping itself stands — it is the substance of the callout. --- skills/stash-postgres/SKILL.md | 19 ++++++++----------- 1 file changed, 8 insertions(+), 11 deletions(-) diff --git a/skills/stash-postgres/SKILL.md b/skills/stash-postgres/SKILL.md index 25eaa60b..f489b946 100644 --- a/skills/stash-postgres/SKILL.md +++ b/skills/stash-postgres/SKILL.md @@ -28,16 +28,14 @@ Supabase client, those integrations emit correct operands for you: see > [CipherStash Proxy](https://github.com/cipherstash/proxy) the split is the > opposite — you write *plaintext* SQL and Proxy encrypts on write and > decrypts on read, so there is no `encryptQuery`, no `eql_v3.query_*` cast, -> and no payload to bind. `stash init` already asked which path you are on -> (`--proxy` / `--no-proxy`) and recorded the answer as `usesProxy` in -> `.cipherstash/context.json` — read it there if you are unsure. +> and no payload to bind. > -> Proxy also has its own schema lifecycle — `stash db push` into -> `eql_v2_configuration`, promoted at cutover — and that lifecycle is the EQL -> **v2** one. This skill is EQL v3, where a column's encryption config lives -> in its domain and there is no configuration table to push (`stash db push` -> against a v3 database says exactly that and refuses). See `stash-encryption` -> § If you use CipherStash Proxy for the Proxy-side rollout. +> Proxy's schema lifecycle — `stash db push` into `eql_v2_configuration`, +> promoted at cutover — belongs to EQL **v2**. This skill is EQL v3, where a +> column's encryption config lives in its own domain and there is no +> configuration table to push. The `stash` CLI does not track a +> Proxy-versus-SDK choice; it targets the direct-connection path this skill +> describes. ## When to Use This Skill @@ -408,8 +406,7 @@ SELECT column_name, domain_schema, domain_name `EXPLAIN` checklist. - `stash-edge` — the WASM entry, `CS_*` credentials, and the credential-identity rule. -- `stash-cli` — `stash eql install`, `stash db validate`, `stash encrypt backfill`, - and the `--proxy` / `--no-proxy` choice behind `usesProxy`. +- `stash-cli` — `stash eql install`, `stash db validate`, `stash encrypt backfill`. Upstream: From c9bc70d20fe21ee1f780ae2b362db559c8f6e681 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 25 Jul 2026 20:20:22 +1000 Subject: [PATCH 5/9] docs(skills): separate the database domain from its language mappings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "Assume `users.email` is a `types.TextEq` column" named the schema builder as though it were the column's type. It isn't: `types.TextEq` is a TypeScript factory in this repo, while the column is `public.eql_v3_text_eq`, a Postgres domain over jsonb. The distinction is the one the whole section turns on, so opening by conflating them was the wrong footing. Now opens with the database type and states that the domain is the authority — what the CHECK enforces and what decides the operator set — then lists the three things that map onto it: the `types.*` schema factory, the `TextEq` / `TextEqQuery` wire types from `@cipherstash/eql`, and the `eql_v3.query_*` operand domain. Records where the TypeScript types come from: generated with the JSON Schemas from the Rust `eql-bindings` crate, with the SQL bundle built from the same commit, so the wire shape and the domain CHECK cannot drift. Verified against @cipherstash/eql 3.0.2 — `TextEq` is `{ v, i, c, hm }` and `TextEqQuery` is `{ v, i, hm }`, so the generated pair is exactly the storage-vs-query split this section goes on to explain. --- skills/stash-postgres/SKILL.md | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/skills/stash-postgres/SKILL.md b/skills/stash-postgres/SKILL.md index f489b946..c7adddc4 100644 --- a/skills/stash-postgres/SKILL.md +++ b/skills/stash-postgres/SKILL.md @@ -47,9 +47,29 @@ Supabase client, those integrations emit correct operands for you: see ## The Two Halves -Assume `users.email` is a `types.TextEq` column — the domain names below -follow the column, so a `types.TextSearch` column would use -`eql_v3.query_text_search` in exactly the same places. +Assume `users.email` is declared in the database as `public.eql_v3_text_eq` — +a Postgres **domain over `jsonb`**. The domain is the authority: it is what +the column actually *is*, what the CHECK constraint enforces, and what decides +which operators the column admits. + +Everything else is a mapping onto that domain, and the names rhyme closely +enough to blur together: + +- **`types.TextEq('email')`** — the schema factory from + `@cipherstash/stack/eql/v3` that *declares* the column. A TypeScript builder, + not the column's type. +- **`TextEq` / `TextEqQuery`** — the wire shapes, exported as TypeScript types + by `@cipherstash/eql`. These and the JSON Schemas are generated from the Rust + `eql-bindings` crate, and the SQL bundle is built from the same commit, so + the payload shape and the domain CHECK cannot drift apart. Each type's doc + comment names the domain it maps to and the operators that domain admits. +- **`eql_v3.query_text_eq`** — the *query* domain, also a database type, that a + search needle is cast to. + +That `TextEq` / `TextEqQuery` pairing is precisely the split this section is +about. The domain names below follow the column, so a +`public.eql_v3_text_search` column would use `eql_v3.query_text_search` in +exactly the same places. ```ts // 1. Encrypt for storage — the payload includes the ciphertext (`c`). From b3512008da1b74d04913109f159414c2c1089855 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 25 Jul 2026 20:51:13 +1000 Subject: [PATCH 6/9] docs(skills): send agents to EQL for the current type surface MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The domain and operator tables read as authoritative. They are not — they are a snapshot of a versioned surface defined in `encrypt-query-language`, and nothing in CI catches them drifting when the `@cipherstash/eql` pin moves. Marks them as a snapshot at the point of use, and adds a ranked list of places to confirm current types: the EQL skill first (it ships beside the bundle it documents, so it tracks the installed version), then the generated `@cipherstash/eql` TypeScript types, then the install SQL's CREATE OPERATOR statements, then `SELECT eql_v3.version()` against the database. The middle two need only `node_modules` — the `stash` CLI depends on `@cipherstash/eql` at an exact pin — so the check costs nothing and needs no connection. The EQL skill does not exist yet (cipherstash/encrypt-query-language#422), so it is referenced by role rather than by a name that may not survive review, and every other rung of the ladder works today without it. --- .changeset/stash-postgres-edge-skills.md | 6 ++- skills/stash-postgres/SKILL.md | 67 +++++++++++++++++------- 2 files changed, 53 insertions(+), 20 deletions(-) diff --git a/.changeset/stash-postgres-edge-skills.md b/.changeset/stash-postgres-edge-skills.md index 9375e148..196d69b7 100644 --- a/.changeset/stash-postgres-edge-skills.md +++ b/.changeset/stash-postgres-edge-skills.md @@ -28,7 +28,11 @@ is otherwise left to infer: **CipherStash Proxy** (where you write plaintext SQL and none of the skill applies — the `usesProxy` fork `stash init` already asked about), and the provenance of the operator surface itself (the EQL bundle from `cipherstash/encrypt-query-language`, version-checkable with -`SELECT eql_v3.version()`, and where operator gaps should be filed). +`SELECT eql_v3.version()`, and where operator gaps should be filed). Its +domain and operator tables are explicitly marked as a snapshot of a versioned +surface, with a ranked list of authorities to confirm current types against — +the EQL skill first, then the generated `@cipherstash/eql` types and install +SQL, both of which need only `node_modules` and no database. **`stash-edge`** — the `@cipherstash/stack/wasm-inline` entry for Deno, Supabase Edge Functions, Cloudflare Workers, and Bun. Import specifier per diff --git a/skills/stash-postgres/SKILL.md b/skills/stash-postgres/SKILL.md index c7adddc4..59cf68ce 100644 --- a/skills/stash-postgres/SKILL.md +++ b/skills/stash-postgres/SKILL.md @@ -52,8 +52,7 @@ a Postgres **domain over `jsonb`**. The domain is the authority: it is what the column actually *is*, what the CHECK constraint enforces, and what decides which operators the column admits. -Everything else is a mapping onto that domain, and the names rhyme closely -enough to blur together: +Everything else is a mapping onto that domain: - **`types.TextEq('email')`** — the schema factory from `@cipherstash/stack/eql/v3` that *declares* the column. A TypeScript builder, @@ -131,6 +130,13 @@ by design — there is no query domain and nothing to search server-side. Which operators each column domain accepts against its query domain. Anything not listed does not exist as an encrypted operator. +> **Confirm types against EQL before relying on them.** This table — and every +> other domain/operator table in this skill — is a snapshot of a *versioned* +> surface that is defined elsewhere. Do not treat it as the last word. Consult +> EQL for the precise current types, in the order given under **Where this +> surface is defined** below; the first two checks need nothing but +> `node_modules`. + | Column domain | Operators | Query domain operand | |---|---|---| | `eql_v3__eq`, `eql_v3_text_eq` | `=` `<>` | `query__eq` / `query_text_eq` | @@ -159,20 +165,38 @@ package and developed at [`cipherstash/encrypt-query-language`](https://github.com/cipherstash/encrypt-query-language). The CLI pins an exact version, so a database is only ever on one bundle. -That makes the matrix a snapshot of a *versioned* surface. Check it against -the bundle actually installed before concluding an operator is missing: - -```sql -SELECT eql_v3.version(); -- e.g. '3.0.2' -``` - -If an operator is absent from both this matrix and your installed bundle, -that is an upstream question, not a client-library one — the operator set -lives in `encrypt-query-language`. The bundle is also readable locally — the -`stash` CLI depends on `@cipherstash/eql`, which ships the install SQL at -`dist/sql/cipherstash-encrypt.sql` (under `node_modules`, wherever your -package manager resolves it). That file is the last word on which overloads -exist. +That makes every table in this skill a snapshot of a *versioned* surface. +**Go to EQL for the precise current types rather than trusting these tables +alone** — in this order: + +1. **The EQL skill**, when it is installed. It ships from + `encrypt-query-language` alongside the bundle it documents, so it tracks the + version you actually have, and it is authoritative for domain names, query + domains, operators, and payload shapes. +2. **The generated TypeScript types** in `@cipherstash/eql`. Every per-domain + type's doc comment names its domain and that domain's operators — the + `TextEqQuery` type, for instance, is documented as the + `eql_v3.query_text_eq` equality query operand admitting `=` and `<>`. They + are generated from the same Rust `eql-bindings` commit as the SQL bundle, + so the two cannot disagree. +3. **The install SQL**, shipped at + `@cipherstash/eql/dist/sql/cipherstash-encrypt.sql` (under `node_modules`, + wherever your package manager resolves it). Its `CREATE OPERATOR` + statements are the last word on which overloads exist — each names its + `LEFTARG`, `RIGHTARG`, and implementing `FUNCTION`. +4. **The database itself**, which is the runtime truth and the right check + when a query is failing right now: + + ```sql + SELECT eql_v3.version(); -- which bundle is actually installed + ``` + +The `stash` CLI depends on `@cipherstash/eql` and pins an exact version, so +2 and 3 are available in any project that has the CLI installed, with no +database connection required. + +If an operator is absent from all of these, that is an upstream question, not +a client-library one — the operator set lives in `encrypt-query-language`. ## Binding Parameters: The Driver Rules @@ -430,10 +454,15 @@ SELECT column_name, domain_schema, domain_name Upstream: +- **The EQL skill**, shipped from + [`cipherstash/encrypt-query-language`](https://github.com/cipherstash/encrypt-query-language) + — the authority for the current domain, operator, and payload-shape surface. + Consult it in preference to the tables here whenever it is installed; those + tables are a snapshot, it tracks the bundle. - [`cipherstash/encrypt-query-language`](https://github.com/cipherstash/encrypt-query-language) - — EQL itself: the authoritative definition of every domain, operator, CHECK, - and extractor named in this skill, shipped as `@cipherstash/eql`. Operator - gaps and domain-level bugs belong there, not against the client library. + — EQL itself: the definition of every domain, operator, CHECK, and extractor + named in this skill, shipped as `@cipherstash/eql`. Operator gaps and + domain-level bugs belong there, not against the client library. - [`cipherstash/proxy`](https://github.com/cipherstash/proxy) — CipherStash Proxy, the alternative to this entire skill: plaintext SQL, encryption on the wire. From 4fdad2c02a65fe5fa2478b7415b4e95e5d9f7545 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 25 Jul 2026 21:04:27 +1000 Subject: [PATCH 7/9] docs(skills): stop teaching native-entry bundling in the edge skill MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `stash-edge` named `@cipherstash/protect-ffi` five times. Three of those were configuration for the runtime this skill exists to steer readers away from: the entry-choice table's Node row explained how to externalise it in Next, and the Workers section named it again to say the config was irrelevant. A reader here is not configuring a Node server, so that is the wrong skill to carry it — the bundling guide already owns it, and is now linked instead. Two mentions stay, for different reasons: - The `When to Use` trigger keeps the name. Deploying the default entry to an edge runtime fails with `Cannot find module '@cipherstash/protect-ffi'`, and that string in a deploy log is how an agent should route itself here. - The Deno note keeps `--allow-ffi`, which is Deno's own permission flag rather than the package — a diagnostic that the wrong entry resolved. The intro now says "a Node-API native module" without naming the package; the contrast it draws does not need it. --- skills/stash-edge/SKILL.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/skills/stash-edge/SKILL.md b/skills/stash-edge/SKILL.md index 0ea023cc..c8334286 100644 --- a/skills/stash-edge/SKILL.md +++ b/skills/stash-edge/SKILL.md @@ -5,9 +5,9 @@ description: Run CipherStash encryption on edge and non-Node runtimes with the ` # Encryption on the Edge (WASM entry) -`@cipherstash/stack` has two runtime entries. The default one binds -`@cipherstash/protect-ffi`, a Node-API native module, and must be loaded by -Node's own `require`. **`@cipherstash/stack/wasm-inline` is the entry for +`@cipherstash/stack` has two runtime entries. The default one binds a +Node-API native module and must be loaded by Node's own `require`. +**`@cipherstash/stack/wasm-inline` is the entry for everywhere else** — it carries the WASM build of the same engine as a base64 blob inside the JS, so there is no native binding, no separate `.wasm` fetch, and nothing for a bundler to externalise. @@ -34,7 +34,7 @@ together. | Runtime | Entry | Why | |---|---|---| -| Node server, Next.js server code | `@cipherstash/stack` (+ `/v3`) | Native NAPI is faster; keep `@cipherstash/protect-ffi` external (e.g. Next's `serverExternalPackages`) | +| Node server, Next.js server code | `@cipherstash/stack` (+ `/v3`) | Native NAPI is faster; the native module must be excluded from bundling — see the [bundling guide](https://cipherstash.com/docs/stack/deploy/bundling) | | Supabase Edge Functions | `@cipherstash/stack/wasm-inline` | Deno, V8-only, no native modules | | Cloudflare Workers | `@cipherstash/stack/wasm-inline` | V8 isolate, no native modules | | Deno (any) | `@cipherstash/stack/wasm-inline` | No NAPI under Deno's default permissions | @@ -99,8 +99,8 @@ import { Encryption, encryptedTable, types } from '@cipherstash/stack/wasm-inlin ``` No `externals`, no `nodeExternals`, no `serverExternalPackages` entry. If a -build config already externalises `@cipherstash/protect-ffi` for the native -entry, that config does not apply here and can be left alone. +build config already externalises the native module for the default entry, +that config does not apply here and can be left alone. ## Credentials From 51bb1303c0a2975eea4344e011a47e323b3c2747 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 25 Jul 2026 21:38:51 +1000 Subject: [PATCH 8/9] docs(skills): correct the lock-context explanation in stash-edge MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DEPENDS ON #797 — do not merge #777 until the wasm-inline `.withLockContext()` lands. This describes the post-#797 surface. The skill said "Identity-bound encryption is configured, not chained" and presented `config.authStrategy` as the replacement for `.withLockContext()`. That conflates two orthogonal mechanisms. An auth strategy decides who the client is; a lock context decides which key the value is encrypted under. You need both, and a strategy alone writes data that is not identity-bound. What actually changed on the native entry was authentication: per-operation CTS tokens were removed in protect-ffi 0.25, so `LockContext.identify()` is deprecated and the strategy handles token acquisition. `.withLockContext()` never went anywhere. The WASM entry picked up the authentication half and not the key-binding half, and the skill then described the gap as though the first subsumed the second. Rewrites the section as two numbered steps, adds the encrypt-side example, and states the symmetry rule — the same claim must be supplied on decrypt, and a mismatch surfaces as a failed decrypt rather than a key error. The code example assumes the chainable form, matching native. If #797 lands a per-call option instead, this example must change before #777 merges. --- skills/stash-edge/SKILL.md | 31 +++++++++++++++++++++++++------ 1 file changed, 25 insertions(+), 6 deletions(-) diff --git a/skills/stash-edge/SKILL.md b/skills/stash-edge/SKILL.md index c8334286..c38ddd03 100644 --- a/skills/stash-edge/SKILL.md +++ b/skills/stash-edge/SKILL.md @@ -237,15 +237,23 @@ Available: `encrypt`, `decrypt`, `isEncrypted`, `encryptQuery`, | Config | discovered from env / `~/.cipherstash` | all four `CS_*` passed explicitly | | Typing | signatures derived from the schema | schema-aware, but not the full typed client | | `.audit()` | chainable on operations | **not available** | -| `.withLockContext()` | chainable on operations | **not available** — see below | +| `.withLockContext()` | chainable on operations | chainable on operations — see below | | `bulkEncrypt` shape | `(plaintexts, { table, column })`, `{ id, plaintext }` envelopes | per-item `{ plaintext, table, column }`, plain index-aligned array | | Module format | ESM + CJS | **ESM only** | -**Identity-bound encryption is configured, not chained.** There is no -`.withLockContext()` on this entry. Build an `OidcFederationStrategy` (or -`AccessKeyStrategy` for service-to-service) and pass it as -`config.authStrategy`, so the client is authenticated *as the end user* for -its whole lifetime: +**Authentication and key binding are two different things**, and conflating +them is the standard mistake. Identity-bound encryption needs both: + +1. **Authenticate as the user** — build an `OidcFederationStrategy` (or + `AccessKeyStrategy` for service-to-service) and pass it as + `config.authStrategy`. The client then acts as that user for its lifetime. +2. **Bind the data key to a claim** — chain `.withLockContext({ identityClaim })` + on the operation. *This* is what changes key derivation. + +An auth strategy on its own gives you a client authenticated as the end user, +writing data that is **not** identity-bound. The strategy replaced the old +per-operation token ceremony (`LockContext.identify()`, deprecated); it did +not replace the lock context. ```ts import { Encryption, OidcFederationStrategy } from '@cipherstash/stack/wasm-inline' @@ -262,8 +270,19 @@ const client = await Encryption({ schemas: [users], config: { authStrategy: strategy.data, clientId, clientKey }, }) + +// Step 2: bind the data key to a claim. Without this the value is encrypted +// under the workspace key, not the user's. +const enc = await client + .encrypt('alice@example.com', { table: users, column: users.email }) + .withLockContext({ identityClaim: ['sub'] }) ``` +**The same claim must be supplied on decrypt.** A value encrypted under a lock +context and decrypted without one — or under a different claim — does not come +back. This is the single most common identity-aware encryption bug, and it +does not surface as a key error; it surfaces as a failed decrypt. + `AccessKeyStrategy.create(workspaceCrn, accessKey)` has the same Result-returning shape, for service-to-service use with a custom token store. When you pass an auth strategy, do **not** also pass `config.accessKey` — they From 94ba0320668e2746565d9bc4e08c4c60fbcdb1c2 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 25 Jul 2026 22:04:57 +1000 Subject: [PATCH 9/9] docs(skills): address CodeRabbit review on #777 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five of six findings applied; the sixth skipped with a reason. - `stash-drizzle`: `db.execute(sql\`…\`)` used backslash-escaped backticks inside an inline code span. Escapes do not work inside code spans, so the backslashes rendered literally. Switched to a double-backtick span, which is the actual mechanism for embedding a backtick. - `stash-edge`, `stash-postgres`: language tags on three unlabelled fences — a TypeScript error, the domain-name mapping, and a Postgres CHECK-violation message. All `text`. - `stash-postgres` frontmatter: the operator list omitted `<=` and `>`, which the predicate matrix includes. The description is the skill-selection surface, so an incomplete list there is worse than verbose. - `stash-postgres` Query Recipes: the recipes dereference `.data` without guarding `.failure`, and agents copy from them. Rather than adding the guard to six short recipes, the section now states the omission up front and shows the guard once — unguarded, the failure surfaces as a domain CHECK violation rather than as the encryption error it is, which is the confusing outcome this skill exists to prevent. Skipped: adding a "the docs site needs a corresponding update" note to the changeset. Changesets become CHANGELOG entries, which are customer-facing; an internal follow-up action does not belong there. Tracked separately. Two other unlabelled fences exist in `stash-encryption` and `stash-cli` but predate this branch and are untouched by it. --- skills/stash-drizzle/SKILL.md | 2 +- skills/stash-edge/SKILL.md | 2 +- skills/stash-postgres/SKILL.md | 17 ++++++++++++++--- 3 files changed, 16 insertions(+), 5 deletions(-) diff --git a/skills/stash-drizzle/SKILL.md b/skills/stash-drizzle/SKILL.md index bc743651..d9d03b22 100644 --- a/skills/stash-drizzle/SKILL.md +++ b/skills/stash-drizzle/SKILL.md @@ -417,7 +417,7 @@ import { index } from "drizzle-orm/pg-core" Run `ANALYZE
` after the migration applies — an expression index gathers no statistics at `CREATE INDEX` time. For when to create indexes during a rollout (after backfill, before switching reads), engagement rules, and `EXPLAIN` verification, see the `stash-indexing` skill. -> **Dropping to raw SQL?** `db.execute(sql\`…\`)` bypasses the operators this integration emits, so you own the operand casts yourself — an encrypted predicate needs its needle cast to the column's `eql_v3.query_*` domain, and the driver's parameter-binding rules differ between `pg` and `postgres-js`. The `stash-postgres` skill is the reference for both. +> **Dropping to raw SQL?** ``db.execute(sql`…`)`` bypasses the operators this integration emits, so you own the operand casts yourself — an encrypted predicate needs its needle cast to the column's `eql_v3.query_*` domain, and the driver's parameter-binding rules differ between `pg` and `postgres-js`. The `stash-postgres` skill is the reference for both. ## Migrating an Existing Column to Encrypted diff --git a/skills/stash-edge/SKILL.md b/skills/stash-edge/SKILL.md index c38ddd03..bcc45754 100644 --- a/skills/stash-edge/SKILL.md +++ b/skills/stash-edge/SKILL.md @@ -313,7 +313,7 @@ round trip per call. A schema authored with `@cipherstash/stack/v3` **will not typecheck** against the WASM entry's `Encryption`, and the reverse fails too: -``` +```text Type 'EncryptedTextSearchColumn' is not assignable to type 'AnyEncryptedV3Column'. Types have separate declarations of a private property 'columnName'. ``` diff --git a/skills/stash-postgres/SKILL.md b/skills/stash-postgres/SKILL.md index 59cf68ce..8e3260b3 100644 --- a/skills/stash-postgres/SKILL.md +++ b/skills/stash-postgres/SKILL.md @@ -1,6 +1,6 @@ --- name: stash-postgres -description: Query EQL v3 encrypted columns from hand-written Postgres SQL over `pg` (node-postgres) or `postgres` (postgres-js) — no ORM. Covers the column-domain-to-query-domain operator matrix (which of `=`, `<>`, `<`, `>=`, `@@`, `@>` each encrypted domain accepts), minting search needles with `encryptQuery`, the per-driver parameter-binding rules for encrypted payloads, and the double-encoding failure that trips the domain CHECK with a message naming neither JSON nor encoding. Use when writing INSERT/SELECT against an encrypted column without an ORM, when a predicate returns zero rows or raises "operator does not exist", or when a domain CHECK constraint rejects an encrypted value on write. Assumes a direct Postgres connection with client-side encryption — CipherStash Proxy encrypts on the wire and needs none of this. +description: Query EQL v3 encrypted columns from hand-written Postgres SQL over `pg` (node-postgres) or `postgres` (postgres-js) — no ORM. Covers the column-domain-to-query-domain operator matrix (which of `=`, `<>`, `<`, `<=`, `>`, `>=`, `@@`, `@>` each encrypted domain accepts), minting search needles with `encryptQuery`, the per-driver parameter-binding rules for encrypted payloads, and the double-encoding failure that trips the domain CHECK with a message naming neither JSON nor encoding. Use when writing INSERT/SELECT against an encrypted column without an ORM, when a predicate returns zero rows or raises "operator does not exist", or when a domain CHECK constraint rejects an encrypted value on write. Assumes a direct Postgres connection with client-side encryption — CipherStash Proxy encrypts on the wire and needs none of this. --- # Raw Postgres SQL Against Encrypted Columns (EQL v3) @@ -99,7 +99,7 @@ be explicit on multi-index domains like `types.TextSearch`. Strip `public.`, insert `query_`, move to the `eql_v3` schema: -``` +```text public.eql_v3_text_eq → eql_v3.query_text_eq public.eql_v3_text_search → eql_v3.query_text_search public.eql_v3_bigint_ord → eql_v3.query_bigint_ord @@ -239,7 +239,7 @@ silently breaks if the project ever moves to `postgres-js`. `${JSON.stringify(payload)}::jsonb` on postgres-js produces: -``` +```text value for domain eql_v3_text_search violates check constraint "eql_v3_text_search_check" ``` @@ -260,6 +260,17 @@ running `jsonb_typeof` on the parameter; `'string'` means double-encoded. Assume `sql` is a postgres-js tag; for `pg` use numbered placeholders as above. +**The recipes below omit the `Result` guard for brevity — your code must not.** +`encryptQuery` returns `{ data } | { failure }`, so reading `.data` without +first checking `.failure` binds `undefined` into the query, which fails as a +domain CHECK violation rather than as the encryption error it actually is. +Every recipe should be read as though it were written: + +```ts +const term = await client.encryptQuery(/* … */) +if (term.failure) throw new Error(term.failure.message) +``` + ### Equality ```ts