next: persist permanent collection deletion denial facts

architecture.d/permanent-deletion-foundation.json

@@ -0,0 +1,9 @@
+{
+  "reason": "One internal database boundary replaces live-row absence with permanent denial facts outside user/collection cascades. Immutable CP intent and independently observed native denial evidence are kept distinct; union cannot erase a higher/local denial. Four internal exports and one relative type import; no endpoint, new configuration/key/signing authority or native protocol surface. This foundation does not open startup gates, acknowledge Deleted, or perform purge; those integrations require the independently current nil registry and typed Gone contract.",
+  "growth": {
+    "productionFiles": 1,
+    "relativeImports": 1,
+    "typeScriptExportDeclarations": 4,
+    "services/server": 1
+  }
+}

changelog.d/permanent-deletion-foundation.md

@@ -0,0 +1,3 @@
+## Added
+
+- Add permanent next-collection deletion denial facts outside live-row/account cascades, with immutable CP intent and denial-monotonic native-floor union primitives. Native fence receipts, startup reconciliation and user-facing deletion integration remain separate work.

docs/permanent-collection-deletion-foundation.md

@@ -0,0 +1,23 @@
+# Permanent collection-deletion denial foundation
+
+Migration `0061_next_collection_deletion_facts.sql` deliberately has no foreign keys to live collection, user, grant or job rows. Removing those rows cannot erase denial. It is the CP journal/denial component only: it does not accept a user deletion, call native deletion, open a startup gate, acknowledge `Deleted`, or purge content.
+
+`collection-deletion.ts` has three internal consumers-to-be:
+
+- `recordCollectionDeletionIntent`: called only in an already authorized/confirmed owner transaction holding the existing collection lock. It chooses the first immutable CP deletion identity and terminal lifecycle epoch1 (previous untracked lifecycle0), retaining actor/time metadata. An existing independently observed floor is returned without creating another CP intent. Retries cannot replace the first CP intent. Returning a local fact is not a native receipt.
+- `mergeCollectionDeletionFloors`: called only after authenticating and strictly validating a current nil-registry page against the configured log-service environment. Bounded128-row pages are copied/validated before any insert. Caller commits a page atomically. Union retains local, older and higher denial evidence; conflicting identities never become optimistic receipt success. The native authority still owns exactly one immutable tuple per collection; retaining conflicting restored CP evidence does not create native successors.
+- `requireCollectionNotDeleted`: presence of ANY fact denies that UUID, independent of live rows. It is not a liveness proof when no fact exists. Final publication callers must integrate it with their existing lock/currentness transaction, not check once before network awaits.
+
+Lifecycle epochs are independent of policy-key epochs, native wakes, fault generation and migration run IDs. PostgreSQL numeric(20,0) retains the full positive u64; code accepts checked bigint and passes its decimal string, never a lossy JS number. SQL-to-Node reads use decimal text. The database is one configured environment; no caller-selected environment or source URL is added.
+
+## Remaining integration
+
+Coordinator-approved native contract is strict typed-only, one immutable `(collection,deletion_id,lifecycle_epoch)` ever. Independent permanent nil-registry floor precedes collection-log Gone, hosted retirement and physical purge. `Deleted` requires BOTH matching typed floor and Gone receipts; a bool, 404, missing row or CP timestamp is insufficient. Native delete_log must independently inspect the durable matching nil floor before serialized Gone.
+
+Before a restored CP serves keys, imports or routes, it must union the NONRESTORED nil registry into this ledger and finish an authenticated complete/current-generation scan. Lost/partial/stale/conflicting scans remain unavailable. Existing tupleless Gone also denies even without a registry/CP fact; exact terminal-status API is required for that check. The CP database/signing key cannot certify itself latest after restore. No new signing authority is added. A completed startup scan is not a later effect-time liveness lease.
+
+Next work wires native record/page/typed Gone/status transport, the fail-closed startup union gate and denial into every registration/key/route/migration/restore path, then existing owner/account deletion and durable cleanup fanout. These primitives alone provide none of those gates or operations.
+
+## Qualification
+
+Disposable PostgreSQL tests cover concurrent first intent, immutable retries, live user/collection cascades, MAXu64, union/idempotency/conflict retention, malformed full-page rejection and surrounding transaction rollback. They do not perform native deletion, restore, hosted retirement or service operations.

services/server/migrations/0061_next_collection_deletion_facts.sql

@@ -0,0 +1,15 @@
+-- Permanent denial facts, deliberately outside every user/collection FK cascade.
+-- CP intent and independently observed nil-registry floors are separate evidence.
+-- These rows are not native Gone receipts or permissions to serve.
+CREATE TABLE next_collection_deletion_facts (
+  collection_id uuid NOT NULL CHECK (collection_id <> '00000000-0000-0000-0000-000000000000'),
+  deletion_id uuid NOT NULL CHECK (deletion_id <> '00000000-0000-0000-0000-000000000000'),
+  lifecycle_epoch numeric(20,0) NOT NULL CHECK (lifecycle_epoch > 0 AND lifecycle_epoch <= 18446744073709551615),
+  authority text NOT NULL CHECK (authority IN ('cp-intent','native-registry')),
+  recorded_at timestamptz NOT NULL DEFAULT now(),
+  actor_id uuid,
+  PRIMARY KEY (collection_id,deletion_id,lifecycle_epoch,authority)
+);
+-- One CP-selected intent ever; independent native facts can only add denial.
+CREATE UNIQUE INDEX next_collection_deletion_first_intent
+  ON next_collection_deletion_facts(collection_id) WHERE authority='cp-intent';

services/server/src/features/next/collection-deletion.postgres.test.ts

@@ -0,0 +1,78 @@
+import { randomUUID } from "node:crypto";
+import pg from "pg";
+import { afterAll, beforeAll, describe, expect, it } from "vitest";
+import { createDatabase, type DatabaseConnection, type DatabasePool } from "../../db.js";
+import { mergeCollectionDeletionFloors, recordCollectionDeletionIntent, requireCollectionNotDeleted, type CollectionDeletionFact } from "./collection-deletion.js";
+
+const testUrl = process.env.MDBASE_CONNECT_TEST_DATABASE_URL;
+const approved = process.env.MDBASE_CONNECT_DESTRUCTIVE_TEST_APPROVAL === "I APPROVE MDBASE CONNECT DESTRUCTIVE POSTGRES TESTS";
+const describePg = testUrl && approved ? describe : describe.skip;
+const max = (1n << 64n) - 1n;
+const fact = (epoch = 1n): CollectionDeletionFact => ({collection:randomUUID(),deletionId:randomUUID(),lifecycleEpoch:epoch});
+describePg("permanent CP deletion denial facts (dedicated local Postgres, no native effects)", () => {
+  let db: DatabasePool, admin: pg.Pool, schema: string;
+  beforeAll(async () => {
+    const url = new URL(testUrl!);
+    if (!["localhost","127.0.0.1","::1"].includes(url.hostname) || !/test/i.test(url.pathname)) throw new Error("Dedicated local test Postgres required");
+    schema = `collection_deletion_${randomUUID().replaceAll("-","")}`;
+    admin = new pg.Pool({connectionString:url.toString(),max:2});
+    await admin.query(`CREATE SCHEMA "${schema}"`);
+    url.searchParams.set("options",`-csearch_path=${schema}`);
+    db = await createDatabase(url.toString());
+  },60000);
+  afterAll(async () => {await db?.end();if(admin&&schema)await admin.query(`DROP SCHEMA "${schema}" CASCADE`);await admin?.end();});
+  async function tx<T>(work:(client:DatabaseConnection)=>Promise<T>):Promise<T>{
+    const client=await db.connect();try{await client.query("BEGIN");const value=await work(client);await client.query("COMMIT");return value;}
+    catch(error){await client.query("ROLLBACK");throw error;}finally{client.release();}
+  }
+  const rows=async(collection:string)=>(await db.query("SELECT deletion_id,lifecycle_epoch::text AS epoch,authority,actor_id FROM next_collection_deletion_facts WHERE collection_id=$1 ORDER BY lifecycle_epoch",[collection])).rows;
+  it("fresh UUID is not denied; first CP intent is permanent and exact retries retain identity/epoch/actor",async()=>{
+    const collection=randomUUID(),actor=randomUUID();await requireCollectionNotDeleted(db,collection);
+    const first=await tx(c=>recordCollectionDeletionIntent(c,collection,actor));
+    expect(first).toMatchObject({collection,lifecycleEpoch:1n});
+    expect(await tx(c=>recordCollectionDeletionIntent(c,collection,randomUUID()))).toEqual(first);
+    await expect(requireCollectionNotDeleted(db,collection)).rejects.toThrow("collection_deleted");
+    expect(await rows(collection)).toEqual([{deletion_id:first.deletionId,epoch:"1",authority:"cp-intent",actor_id:actor}]);
+  });
+  it("concurrent first requests leave exactly one immutable CP intent",async()=>{
+    const collection=randomUUID();const requests=await Promise.all(Array.from({length:8},()=>tx(c=>recordCollectionDeletionIntent(c,collection,randomUUID()))));
+    expect(new Set(requests.map(r=>r.deletionId)).size).toBe(1);expect(await rows(collection)).toHaveLength(1);
+  });
+  it("intent/floor survive real user and live next/hosted row cascades",async()=>{
+    const actor=randomUUID(),collection=randomUUID();await db.query("INSERT INTO users(id,email,name) VALUES($1,$2,'Test')",[actor,`${actor}@example.test`]);
+    await db.query("INSERT INTO hosted_collections(id,user_id,display_name,template) VALUES($1,$2,'Test','mdbase')",[collection,actor]);
+    await db.query("INSERT INTO next_collections(collection_id,owner_user_id,runtime,sync,root_key_id) VALUES($1,$2,'next','cloud_copy',$3)",[collection,actor,Buffer.alloc(16)]);
+    const intent=await tx(c=>recordCollectionDeletionIntent(c,collection,actor));await tx(c=>mergeCollectionDeletionFloors(c,[intent]));
+    await db.query("DELETE FROM users WHERE id=$1",[actor]);expect(await rows(collection)).toHaveLength(2);
+    await expect(requireCollectionNotDeleted(db,collection)).rejects.toThrow("collection_deleted");
+  });
+  it("preserves MAXu64 exactly, repeats idempotently and retains higher/local/conflicting denial facts",async()=>{
+    const floor=fact(max);const local=await tx(c=>recordCollectionDeletionIntent(c,floor.collection,randomUUID()));
+    await tx(c=>mergeCollectionDeletionFloors(c,[floor]));await tx(c=>mergeCollectionDeletionFloors(c,[floor]));
+    await tx(c=>mergeCollectionDeletionFloors(c,[{...floor,deletionId:randomUUID(),lifecycleEpoch:2n}]));
+    expect((await rows(floor.collection)).map(r=>r.epoch)).toEqual(["1","2",max.toString()]);
+    expect((await rows(floor.collection)).some(r=>r.deletion_id===local.deletionId)).toBe(true);
+    await expect(requireCollectionNotDeleted(db,floor.collection)).rejects.toThrow("collection_deleted");
+  });
+  it("an already observed native floor supplies no fabricated CP intent",async()=>{
+    const floor=fact(7n);await tx(c=>mergeCollectionDeletionFloors(c,[floor]));
+    expect(await tx(c=>recordCollectionDeletionIntent(c,floor.collection,randomUUID()))).toEqual(floor);
+    expect(await rows(floor.collection)).toHaveLength(1);
+  });
+  it("validates the entire page before any writes, including duplicate collections/u64/non-nil UUID",async()=>{
+    const good=fact();const bad=[{...fact(),lifecycleEpoch:0n},{...fact(),lifecycleEpoch:max+1n},{...fact(),deletionId:"00000000-0000-0000-0000-000000000000"}];
+    for(const value of bad){await expect(tx(c=>mergeCollectionDeletionFloors(c,[good,value]))).rejects.toThrow();expect(await rows(good.collection)).toHaveLength(0);}
+    await expect(tx(c=>mergeCollectionDeletionFloors(c,[good,{...good,deletionId:randomUUID()}]))).rejects.toThrow("duplicate_collection_deletion_floor");
+    await expect(tx(c=>mergeCollectionDeletionFloors(c,Array.from({length:129},()=>fact())))).rejects.toThrow("invalid_collection_deletion_page");
+    expect(await rows(good.collection)).toHaveLength(0);
+  });
+  it("failed surrounding transaction leaves no acknowledged intent/page effects",async()=>{
+    const value=fact();await expect(tx(async c=>{await mergeCollectionDeletionFloors(c,[value]);throw new Error("synthetic crash before commit");})).rejects.toThrow("synthetic crash");
+    await requireCollectionNotDeleted(db,value.collection);expect(await rows(value.collection)).toHaveLength(0);
+  });
+  it("DB constraints reject nil identity and out-of-range epochs; no cascade references exist",async()=>{
+    for(const epoch of ["0","18446744073709551616"]){await expect(db.query("INSERT INTO next_collection_deletion_facts(collection_id,deletion_id,lifecycle_epoch,authority) VALUES($1,$2,$3,'native-registry')",[randomUUID(),randomUUID(),epoch])).rejects.toThrow();}
+    await expect(db.query("INSERT INTO next_collection_deletion_facts(collection_id,deletion_id,lifecycle_epoch,authority) VALUES($1,$2,1,'native-registry')",["00000000-0000-0000-0000-000000000000",randomUUID()])).rejects.toThrow();
+    const constraints=await db.query("SELECT constraint_name FROM information_schema.table_constraints WHERE table_schema=$1 AND table_name='next_collection_deletion_facts' AND constraint_type='FOREIGN KEY'",[schema]);expect(constraints.rows).toEqual([]);
+  });
+});

services/server/src/features/next/collection-deletion.test.ts

@@ -0,0 +1,51 @@
+import { randomUUID } from "node:crypto";
+import { describe, expect, it, vi } from "vitest";
+import type { DatabaseConnection } from "../../database-types.js";
+import { mergeCollectionDeletionFloors, type CollectionDeletionFact } from "./collection-deletion.js";
+
+const fact = (): CollectionDeletionFact => ({collection:randomUUID(),deletionId:randomUUID(),lifecycleEpoch:(1n<<64n)-1n});
+function peer(onWrite = () => {}) {
+  const query = vi.fn(async (_text: string, _values?: unknown[]) => {
+    onWrite(); return {rows:[],rowCount:1,command:"INSERT",oid:0,fields:[]};
+  });
+  const client: DatabaseConnection = {query,release:()=>{}};
+  return {client,query};
+}
+describe("bounded deletion DTO admission (mock database, no native effects)",()=>{
+  it("captures only the three validated primitive fields, never unrelated getters",async()=>{
+    const value=fact(), original={...value}, reads={collection:0,deletionId:0,lifecycleEpoch:0};
+    const input={
+      get collection(){reads.collection++;return value.collection;},
+      get deletionId(){reads.deletionId++;return value.deletionId;},
+      get lifecycleEpoch(){reads.lifecycleEpoch++;return value.lifecycleEpoch;},
+      get unrelated(){throw new Error("unbounded extra property must not be copied");},
+    };
+    const {client,query}=peer(()=>{value.collection=randomUUID();value.deletionId=randomUUID();value.lifecycleEpoch=1n;});
+    await mergeCollectionDeletionFloors(client,[input]);
+    expect(reads).toEqual({collection:1,deletionId:1,lifecycleEpoch:1});
+    expect(query.mock.calls[0][1]).toEqual([original.collection,original.deletionId,original.lifecycleEpoch.toString()]);
+  });
+  it("snapshots every row before the first database await, preserving the full u64",async()=>{
+    const input=[fact(),fact()], originals=input.map(v=>({...v}));
+    const {client,query}=peer(()=>{for(const v of input){v.collection=randomUUID();v.deletionId=randomUUID();v.lifecycleEpoch=1n;}});
+    await mergeCollectionDeletionFloors(client,input);
+    expect(query.mock.calls.map(call=>call[1])).toEqual(originals.map(v=>[v.collection,v.deletionId,v.lifecycleEpoch.toString()]));
+  });
+  it("refuses non-primitive/coercible UUIDs without coercion or any writes",async()=>{
+    const stringify=vi.fn(()=>randomUUID());
+    const {client,query}=peer();
+    for(const invalid of [{toString:stringify},36,null,undefined,new String(randomUUID())]){
+      const value={...fact(),collection:invalid} as unknown as CollectionDeletionFact;
+      await expect(mergeCollectionDeletionFloors(client,[fact(),value])).rejects.toThrow("invalid_collection_deletion_uuid");
+    }
+    expect(stringify).not.toHaveBeenCalled();expect(query).not.toHaveBeenCalled();
+  });
+  it("refuses noncanonical length, nil and alternate UUID text before any writes",async()=>{
+    const {client,query}=peer(), id=randomUUID();
+    for(const invalid of [id+"\n",id+" ",` ${id}`,id.replaceAll("-",""),"00000000-0000-0000-0000-000000000000","é".repeat(36)]){
+      const value={...fact(),deletionId:invalid};
+      await expect(mergeCollectionDeletionFloors(client,[fact(),value])).rejects.toThrow("invalid_collection_deletion_uuid");
+    }
+    expect(query).not.toHaveBeenCalled();
+  });
+});