diff --git a/CHANGELOG.md b/CHANGELOG.md index 9f2f494..72c70f3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,73 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- **Known limitation (parked, owner: Danny): a host-environment hold can stall the + mirror publish.** On the development host, replacing a freshly written `ledger.json` + intermittently refuses `EPERM` for longer than any bounded retry we ship (measured: + the source tmp stays movable, the destination opens `r+`, the replace alone refuses — + a holder sharing read/write but not delete; ~1–3% of operations, only under sustained + machine load, never reproduced by isolated probes). The canonical publish now waits + against a deadline (`RATCHET_PUBLISH_TIMEOUT_MS`, default 10s, the same shape git + ships for Windows renames), recovery's own mirror publish wears the same retryable + `ERATCHETMIRRORPENDING` code instead of leaking raw `EPERM`, and both doors answer + "re-run the command" — which provably converges (the WAL suite's M4, and its settled + harness). What is NOT claimed: that a single call always succeeds on such a host. + Diagnosing the holder needs OS-level tooling (handle enumeration), which is an + operator investigation, not a code path. + + + +- **The defect transitions ride the slot — the mirror stops being best-effort.** Step + 4b.2: `defect.resolve`, `defect.reopen`, `defect.supersede` join the wire (18-tool + `--write` roster), and every transition on BOTH doors — the CLI-only `defect waive` + included — commits its state change and its ledger-mirror status behind the same + write-ahead intent. The old best-effort mirror sync is gone: a ledger failure now + surfaces instead of being swallowed, and recovery completes the mirror the dying + process proved. **Named behavior change:** an exact-repeat transition (same target + status, same proof fields, mirror valid) is now a no-op — no log line, no history, no + revision — where it used to grow the log on every rerun; a repeat with DIFFERENT + proof refuses rather than silently replacing the original, and an exact repeat over a + missing or ambiguous mirror commits once solely to perform the D2b admission. + `defect.waive` remains absent from `tools/list` and the dispatcher, permanently. Five + new falsifiers (both-door equivalence, exact-repeat and conflicting-repeat semantics, + per-transition D2b admission, a real process dying between a transition and its + mirror, wire replay); the repeat no-op and the mirror-status propagation each seen + red against a deliberately broken variant. + + + +- **The write-ahead intent slot — one operation, two canonical files, one crash story.** + Step 4b.1 of the ratified WAL design + (docs/superpowers/specs/2026-07-31-mcp-4b-wal-design.md): `defect.add` is the first + cross-file verb, on both doors — the state record, its QA-ledger mirror, and the + back-link commit behind one create-exclusive, size-capped, version-1 `intent.json` + carrying MATERIALIZED post-images and four exact pre/post byte hashes. The state + commit is the decision; recovery — living at the shared post-acquire path of BOTH + workspace-lock APIs, so every supported writer inherits it (**CLI-enforced**) — lands + every process death on one of three legal hash pairs and finishes or discards the + work byte-exactly, proven against the hashes the dying process recorded. Anything + strict recovery cannot prove preserves every byte and refuses `MirrorUnrecoverable` + on every write door and on `workspace.open`; `AttachmentAmbiguous` names the + several-live-artifacts refusal; both sentences are spec-pinned literals in the one + funnel. Dedup stays a no-op decided before any intent; escalation reaches the mirror + in the same operation; a legacy defect with no valid mirror is admitted on its first + committed escalation (D2b — one new mirror, old rows untouched). Every read states + `pendingIntent` out loud (race-safe revision+token sampling, derived, never + persisted); `ratchet doctor` gains read-only WAL diagnosis that names the condition + and never repairs. The guarantee covers process/server death, NOT sudden power loss + — file fsync is best-effort and the directory is never fsynced, stated in spec and + code. The canonical publish also gained a brief fenced retry for transient Windows + rename refusals (surfaced by the crash matrix; persistent refusals still throw). 22 + falsifiers in `test/mcp-wal.test.js` including a real-process crash matrix dying at + the intent link, the state rename, the mirror rename, the clear unlink, and twice + INSIDE recovery itself; four mechanisms each seen red against a deliberately broken + variant (choke point disabled → 9 red, hash machine loosened, receipt validation + skipped, publish retry removed). `workspace.open` now snapshots under the workspace + lock so recovery precedes every handle. + + + + - **The artifact verbs and the read that writes — the safe core is complete.** Step 4.3 of the ratified write-tools design: `artifact.add`, `artifact.close`, `artifact.retract` and `score.aperture` complete the ten-tool `--write` roster, each a diff --git a/package.json b/package.json index f9909b1..b186a5e 100644 --- a/package.json +++ b/package.json @@ -15,7 +15,7 @@ "node": ">=18" }, "scripts": { - "test": "node test/cli.test.js && node test/evolve.test.js && node test/plugin-shape.test.js && node test/concurrency.test.js && node test/mcp-rpc.test.js && node test/mcp-workspace.test.js && node test/mcp-handles.test.js && node test/mcp-repository.test.js && node test/mcp-server.test.js && node test/mcp-write.test.js && node test/mcp-prompts.test.js && node test/mcp-toctou.test.js && node test/mcp-entry.test.js", + "test": "node test/cli.test.js && node test/evolve.test.js && node test/plugin-shape.test.js && node test/concurrency.test.js && node test/mcp-rpc.test.js && node test/mcp-workspace.test.js && node test/mcp-handles.test.js && node test/mcp-repository.test.js && node test/mcp-server.test.js && node test/mcp-write.test.js && node test/mcp-wal.test.js && node test/mcp-prompts.test.js && node test/mcp-toctou.test.js && node test/mcp-entry.test.js", "test:concurrency": "node test/concurrency.test.js", "prompts-gen": "node scripts/prompts-gen.js", "preflight": "node scripts/preflight.js", diff --git a/src/artifacts.js b/src/artifacts.js index 1009716..6c34694 100644 --- a/src/artifacts.js +++ b/src/artifacts.js @@ -5,6 +5,7 @@ const schemas = require('./schemas'); const scoring = require('./scoring'); const lifecycle = require('./lifecycle'); const journal = require('./evolve/journal'); +const wal = require('./wal'); // A domain refusal both boundaries must tell apart from damage carries a code: // the CLI prints the message, the MCP funnel maps the code to its one @@ -212,14 +213,55 @@ function resolveAttachment(cwd, s, item) { const live = lifecycle.liveArtifacts(s); if (live.length === 1) return { artifact: live[0].id, attachedBy: 'auto' }; if (live.length === 0) return { artifact: '', attachedBy: 'none' }; - throw new Error( + throw coded( + 'ERATCHETATTACH', `${live.length} live artifacts (${live.map((a) => `${a.id} "${a.title}"`).join(', ')}) — name the one this ` + 'defect attacks with "artifact": "". An unattached defect drains every artifact and blocks closure for all of them.' ); } -function addDefect(cwd, item, { alsoLedger = true } = {}) { - const s = state.loadState(cwd); +// Ledger mirror ids are collision-checked against the ledger they will enter — +// the derived-id rule extends over the mirror collections (4b spec). +function mintLedgerId(ledger, mintId) { + const id = mintId(schemas.LEDGER_COLLECTIONS.defects, 'ledger'); + if ((ledger.defects || []).some((x) => x && x.id === id)) { + throw coded('ERATCHETIDCONFLICT', `derived ledger id ${id} already names a record`); + } + return id; +} + +// The mirror op for a defect that already lives in state. Exactly one linked +// mirror → replace it in place. Anything else — no link, a link to nothing, a +// duplicated id — is the D2b admission: mint a complete mirror from the +// post-transition defect, back-link it in the SAME state post-image, and leave +// every old row untouched; admission never guesses which row to overwrite. +function mirrorOpFor(ledger, defect, now, mintId) { + const linked = String(defect.ledgerId || ''); + const matches = linked ? (ledger.defects || []).filter((x) => x && x.id === linked) : []; + if (matches.length === 1) { + // The mirror follows the record on both axes the transitions own; carrying + // an unchanged value is a no-op inside the same replace. + const after = { ...matches[0], severity: defect.severity, status: defect.status || 'open', updatedAt: now }; + return { collection: 'defects', id: linked, mode: 'replace', after }; + } + const mirror = { + id: mintLedgerId(ledger, mintId), + at: now, + feature: '', + severity: defect.severity, + summary: defect.summary, + status: defect.status || 'open', + foundAt: defect.at || now, + }; + defect.ledgerId = mirror.id; + return { collection: 'defects', id: mirror.id, mode: 'insert', after: mirror }; +} + +// The 4b defect.add core, shared by the CLI and MCP doors: mutates the open +// transaction's state and returns MATERIALIZED mirror ops — ids, timestamps +// and post-images final before any intent publishes. `ledger` is null only on +// the alsoLedger:false path, which writes one file and owes no mirror. +function prepareDefectAdd(cwd, s, ledger, item, mintId) { const now = schemas.nowIso(); const sev = (item.severity || 'medium').toLowerCase(); const severity = schemas.SEVERITIES.includes(sev) ? sev : 'medium'; @@ -236,33 +278,28 @@ function addDefect(cwd, item, { alsoLedger = true } = {}) { // The same finding reported twice is one defect, not two drains. Match on the // pair that identifies it — which artifact, and what it says — while it is - // still live. A repeat that is worse escalates the record in place. + // still live. A repeat that is worse escalates the record in place; a repeat + // that is not is a no-op decided BEFORE any intent exists. const key = summary.trim().toLowerCase(); const dup = (s.defects || []).find( (d) => d && scoring.isDefectOpen(d) && String(d.artifact || '') === artifact && String(d.summary || '').trim().toLowerCase() === key ); if (dup) { const from = dup.severity; - if (schemas.SEVERITIES.indexOf(severity) < schemas.SEVERITIES.indexOf(from)) { - dup.severity = severity; - dup.log = Array.isArray(dup.log) ? dup.log : []; - dup.log.push({ at: now, from, to: severity, note: 'severity escalated by a repeat report' }); - s.dirty = true; - s.history.push({ id: state.makeId('hist'), at: now, event: 'defect.escalated', note: `${dup.id}: ${from} → ${severity}` }); - state.saveState(cwd, s); - if (dup.ledgerId) { - try { - require('./ledger').upsert(cwd, 'defects', { id: dup.ledgerId, severity }, { via: 'transition' }); - } catch (_e) { - /* ledger sync is best-effort */ - } - } + if (schemas.SEVERITIES.indexOf(severity) >= schemas.SEVERITIES.indexOf(from)) { + return { kind: 'noop', action: 'deduped', record: dup, result: { state: dup, ledger: null, deduped: true } }; } - return { state: dup, ledger: null, deduped: true }; + dup.severity = severity; + dup.log = Array.isArray(dup.log) ? dup.log : []; + dup.log.push({ at: now, from, to: severity, note: 'severity escalated by a repeat report' }); + s.dirty = true; + s.history.push({ id: mintId('hist', 'history'), at: now, event: 'defect.escalated', note: `${dup.id}: ${from} → ${severity}` }); + const ledgerOps = ledger ? [mirrorOpFor(ledger, dup, now, mintId)] : []; + return { kind: 'commit', action: 'escalated', record: dup, ledgerOps, result: { state: dup, ledger: null, deduped: true } }; } const record = { - id: item.id || state.makeId('def'), + id: item.id || mintId('def', 'record'), at: now, severity, summary, @@ -290,27 +327,50 @@ function addDefect(cwd, item, { alsoLedger = true } = {}) { record.attachError = e && e.message ? e.message : String(e); } } - s.defects.push(record); - s.dirty = true; - s.history.push({ id: state.makeId('hist'), at: now, event: 'defect.add', note: `[${record.severity}] ${record.summary}` }); - state.saveState(cwd, s); - - let ledgerRecord = null; - if (alsoLedger) { - const ledger = require('./ledger'); - ledgerRecord = ledger.upsert(cwd, 'defects', { + let mirror = null; + const ledgerOps = []; + if (ledger) { + mirror = { + id: mintLedgerId(ledger, mintId), + at: now, feature: item.feature || '', severity: record.severity, summary: record.summary, status: record.status, foundAt: now, - }, { via: 'transition' }).item; - // Link the state defect to its ledger mirror so lifecycle transitions can - // keep both surfaces honest instead of letting the ledger silently drift. - record.ledgerId = ledgerRecord.id; - state.saveState(cwd, s); + }; + // The back-link rides the SAME post-image as the defect — the old second + // state save (and its crash window) is what 4b exists to remove. + record.ledgerId = mirror.id; + ledgerOps.push({ collection: 'defects', id: mirror.id, mode: 'insert', after: mirror }); } - return { state: record, ledger: ledgerRecord }; + s.defects.push(record); + s.dirty = true; + s.history.push({ id: mintId('hist', 'history'), at: now, event: 'defect.add', note: `[${record.severity}] ${record.summary}` }); + return { kind: 'commit', action: 'created', record, ledgerOps, result: { state: record, ledger: mirror } }; +} + +function addDefect(cwd, item, { alsoLedger = true } = {}) { + if (!alsoLedger) { + // One canonical file — the ordinary boundary; no mirror, no intent. + return state.withWorkspaceMutation(cwd, { action: 'defect add' }, (s) => + prepareDefectAdd(cwd, s, null, item, (prefix) => state.makeId(prefix)).result + ).result; + } + // The CLI door of the 4b protocol. The operation id is an internal WAL id + // and the args hash a diagnostic — a repeated CLI command is made safe by + // recovery plus the dedup no-op, never by pretending it has the old id. + return state.withMirroredMutation(cwd, { + action: 'defect add', + door: 'cli', + tool: 'defect add', + operationId: state.makeId('wal'), + argsHash: wal.hashBytes(Buffer.from(JSON.stringify(item === undefined ? null : item), 'utf8')), + }, (s, ledger) => { + const prep = prepareDefectAdd(cwd, s, ledger, item, (prefix) => state.makeId(prefix)); + if (prep.kind === 'noop') return { kind: 'noop', result: prep.result }; + return { kind: 'commit', ledgerOps: prep.ledgerOps, result: prep.result }; + }).result; } // Move a defect through its lifecycle: open/patched/reopened → resolved | waived @@ -318,7 +378,23 @@ function addDefect(cwd, item, { alsoLedger = true } = {}) { // 0.2: a defect could be born but never cleared, so remediated work stayed // confidence-blocking forever. The scorer already honors terminal statuses // (scoring.isDefectOpen); this is what finally lets a defect *reach* one. -function transitionDefect(cwd, id, toStatus, meta = {}) { +// The proof fields each terminal status records — the identity of a +// transition for the exact-repeat rule: same target, same proof → no-op; +// same target, different proof → refuse, never silently replace. +function transitionProof(d, toStatus, meta) { + if (toStatus === 'resolved') return [String(d.evidence || ''), String(meta.evidence || '')]; + if (toStatus === 'reopened') return [String(d.reopenReason || ''), String(meta.reason || '')]; + if (toStatus === 'waived') { + return [`${d.waivedBy || ''}\n${d.waiveReason || ''}`, `${meta.owner || ''}\n${meta.reason || ''}`]; + } + return [String(d.supersededBy || ''), String(meta.by || '')]; +} + +// The 4b transition core, shared by the CLI (resolve/reopen/waive/supersede) +// and the MCP door (waive stays CLI-only by rule). Mutates the transaction's +// state and returns the materialized mirror op; the exact-repeat no-op is +// decided HERE, before any intent exists. +function prepareDefectTransition(s, ledger, id, toStatus, meta, mintId) { if (toStatus === 'closed') { throw new Error( '"closed" is a read-only legacy alias (pre-0.3) and cannot be transitioned into — it is terminal with no ' + @@ -343,48 +419,77 @@ function transitionDefect(cwd, id, toStatus, meta = {}) { } if (toStatus === 'superseded') need(meta_.by, `cannot supersede defect "${id}": --by must name what replaced it`); - const s = state.loadState(cwd); const d = (s.defects || []).find((x) => x.id === id); - if (!d) throw new Error(`no defect with id "${id}"`); + if (!d) throw coded('ERATCHETUNKNOWNID', `no defect with id "${id}"`); const now = schemas.nowIso(); const from = d.status || 'open'; + if (from === toStatus) { + const [recorded, offered] = transitionProof(d, toStatus, meta_); + const linked = String(d.ledgerId || ''); + const mirrorValid = ledger + ? linked && (ledger.defects || []).filter((x) => x && x.id === linked).length === 1 + : true; + if (recorded === offered && mirrorValid) { + // The exact repeat, mirror already truthful: no log line, no history, no + // revision, no intent. Extends the 0.9 no-op property (named CHANGELOG + // behavior change — repeats used to grow the log). + return { kind: 'noop', record: d }; + } + if (recorded !== offered) { + throw new Error( + `defect "${id}" is already ${toStatus} with different recorded proof — a repeat does not silently ` + + 'replace the original. Reopen it first if the recorded proof is wrong.' + ); + } + // Exact repeat but the mirror is missing or ambiguous: commit once solely + // to perform the D2b admission below. + } + d.status = toStatus; d.log = Array.isArray(d.log) ? d.log : []; - d.log.push({ at: now, from, to: toStatus, note: meta.note || '' }); + d.log.push({ at: now, from, to: toStatus, note: meta_.note || '' }); // Stamp the fields each transition owns; clear stale ones on reopen. if (toStatus === 'resolved') { d.resolvedAt = now; - if (meta.evidence) d.evidence = meta.evidence; + if (meta_.evidence) d.evidence = meta_.evidence; } if (toStatus === 'reopened') { d.resolvedAt = null; - d.reopenReason = meta.reason || ''; + d.reopenReason = meta_.reason || ''; } if (toStatus === 'waived') { - d.waivedBy = meta.owner || ''; - d.waiveReason = meta.reason || ''; + d.waivedBy = meta_.owner || ''; + d.waiveReason = meta_.reason || ''; } if (toStatus === 'superseded') { - d.supersededBy = meta.by || ''; + d.supersededBy = meta_.by || ''; } s.dirty = true; - s.history.push({ id: state.makeId('hist'), at: now, event: `defect.${toStatus}`, note: `${id}: ${from} → ${toStatus}` }); - state.saveState(cwd, s); + s.history.push({ id: mintId('hist', 'history'), at: now, event: `defect.${toStatus}`, note: `${id}: ${from} → ${toStatus}` }); + const ledgerOps = ledger ? [mirrorOpFor(ledger, d, now, mintId)] : []; + return { kind: 'commit', record: d, ledgerOps }; +} - // Keep the QA ledger mirror in step. Best-effort: a defect added before the - // link existed has no mirror to sync, and a ledger hiccup must never strand a - // state transition that already succeeded. - if (d.ledgerId) { - try { - require('./ledger').upsert(cwd, 'defects', { id: d.ledgerId, status: toStatus }, { via: 'transition' }); - } catch (_e) { - /* ledger sync is best-effort */ - } - } - return d; +// Move a defect through its lifecycle. Since 4b.2 the mirror is not +// best-effort: the state transition and its ledger mirror commit behind one +// write-ahead intent, a mirror failure surfaces instead of being swallowed, +// and a defect with no valid mirror is admitted on this first committed +// mutation (D2b). +function transitionDefect(cwd, id, toStatus, meta = {}) { + return state.withMirroredMutation(cwd, { + action: `defect ${toStatus === 'reopened' ? 'reopen' : toStatus === 'resolved' ? 'resolve' : toStatus === 'waived' ? 'waive' : 'supersede'}`, + door: 'cli', + tool: `defect ${toStatus === 'reopened' ? 'reopen' : toStatus === 'resolved' ? 'resolve' : toStatus === 'waived' ? 'waive' : 'supersede'}`, + operationId: state.makeId('wal'), + argsHash: wal.hashBytes(Buffer.from(JSON.stringify([id, toStatus, meta || null]), 'utf8')), + }, (s, ledger) => { + const prep = prepareDefectTransition(s, ledger, id, toStatus, meta, (prefix) => state.makeId(prefix)); + if (prep.kind === 'noop') return { kind: 'noop', result: prep.record }; + return { kind: 'commit', ledgerOps: prep.ledgerOps, result: prep.record }; + }).result; } // Retract an artifact whose claim turned out false or obsolete. Provenance is @@ -537,6 +642,8 @@ function applyClose(cwd, s, id, opts, mintId) { module.exports = { addArtifact, addDefect, + prepareDefectAdd, + prepareDefectTransition, transitionDefect, retractArtifact, assertArtifactInput, diff --git a/src/cli.js b/src/cli.js index d7c354f..f625f63 100644 --- a/src/cli.js +++ b/src/cli.js @@ -459,7 +459,9 @@ function cmdDefect(cwd, argv, asJson) { switch (sub) { case 'add': { const payload = readPayload(positionals[2]); - const rec = mutate(cwd, 'defect add', () => artifacts.addDefect(cwd, payload)); + // No outer transaction: addDefect owns the 4b mirrored boundary (state + + // ledger behind one write-ahead intent), and a boundary cannot nest. + const rec = artifacts.addDefect(cwd, payload); return out(`defect ${rec.state.id} added: [${rec.state.severity}] ${rec.state.summary}`); } case 'list': { @@ -479,14 +481,14 @@ function cmdDefect(cwd, argv, asJson) { // Proof gate, same spirit as the evolve KEEP gate: a defect cannot be // marked fixed without stating the proof that it is actually fixed. need(evidence, 'defect resolve requires --evidence "" — no proof, no resolve'); - mutate(cwd, 'defect resolve', () => artifacts.transitionDefect(cwd, id, 'resolved', { evidence, note: `resolved: ${evidence}` })); + artifacts.transitionDefect(cwd, id, 'resolved', { evidence, note: `resolved: ${evidence}` }); return out(`defect ${id} → resolved`); } case 'reopen': { need(id, 'usage: ratchet defect reopen --reason ""'); const reason = strOpt(opts.reason); need(reason, 'defect reopen requires --reason ""'); - mutate(cwd, 'defect reopen', () => artifacts.transitionDefect(cwd, id, 'reopened', { reason, note: `reopened: ${reason}` })); + artifacts.transitionDefect(cwd, id, 'reopened', { reason, note: `reopened: ${reason}` }); return out(`defect ${id} → reopened`); } case 'waive': { @@ -495,9 +497,7 @@ function cmdDefect(cwd, argv, asJson) { const reason = strOpt(opts.reason); need(owner, 'defect waive requires --owner ""'); need(reason, 'defect waive requires --reason ""'); - mutate(cwd, 'defect waive', () => - artifacts.transitionDefect(cwd, id, 'waived', { owner, reason, note: `waived by ${owner}: ${reason}` }) - ); + artifacts.transitionDefect(cwd, id, 'waived', { owner, reason, note: `waived by ${owner}: ${reason}` }); return out(`defect ${id} → waived (owner: ${owner})`); } case 'supersede': { @@ -505,13 +505,11 @@ function cmdDefect(cwd, argv, asJson) { const by = strOpt(opts.by); need(by, 'defect supersede requires --by '); const reason = strOpt(opts.reason); - mutate(cwd, 'defect supersede', () => - artifacts.transitionDefect(cwd, id, 'superseded', { - by, - reason, - note: `superseded by ${by}${reason ? `: ${reason}` : ''}`, - }) - ); + artifacts.transitionDefect(cwd, id, 'superseded', { + by, + reason, + note: `superseded by ${by}${reason ? `: ${reason}` : ''}`, + }); return out(`defect ${id} → superseded (by: ${by})`); } default: @@ -817,6 +815,27 @@ function cmdDoctor(cwd, asJson) { } add('state dir writable', stateOk, stateDetail); + // 4b WAL slot — read-only diagnosis, never a repair. A recoverable slot is + // informational (any supported write recovers it on its way in); a slot + // strict recovery cannot prove legal is the operator's, by name. + try { + const slot = state.diagnoseIntent(cwd); + if (!slot.pending) add('WAL intent slot', true, 'no pending intent'); + else if (slot.verdict === 'ambiguous') { + add('WAL intent slot', false, + `${slot.reason} Do not delete the slot blindly — repair the named condition; recovery clears it.`); + } else { + const meaning = { + discarded: 'discardable — the state decision never published', + completed: 'mirror owed — the next supported write completes it', + cleared: 'clearable — the mirror already landed', + }; + add('WAL intent slot', true, `pending intent, ${meaning[slot.verdict] || slot.verdict}`); + } + } catch (e) { + add('WAL intent slot', false, e.message); + } + let snapOk = true; let snapDetail = ''; try { diff --git a/src/mcp/ops.js b/src/mcp/ops.js index 76ec9f2..df1c623 100644 --- a/src/mcp/ops.js +++ b/src/mcp/ops.js @@ -141,6 +141,12 @@ const CODED_OUTCOMES = { ERATCHETCLOSUREBLOCKED: 'closureBlocked', ERATCHETHUMANAUTHORITY: 'humanAuthority', ERATCHETRETRACT: 'retractRefused', + // 4b: strict recovery could not prove the store legal / defect attachment + // needs an explicit artifact. ERATCHETMIRRORPENDING is deliberately ABSENT: + // a post-decision mirror failure is not an outcome to answer with — the + // throw funnels to the retryable WriteFailed and the retry recovers first. + ERATCHETMIRROR: 'mirror', + ERATCHETATTACH: 'attachAmbiguous', }; function executeWrite(opts) { @@ -229,6 +235,80 @@ function executeWrite(opts) { return outcome; } +// The cross-file variant (4b): the same ordered replay → generation → revision +// checks, but the mutation runs through state.withMirroredMutation, so the +// ledger half rides a write-ahead intent and a death between the two files is +// a recoverable lag. `prepare(s, ledger, mintId)` returns null after recording +// a refusal outcome, { kind: 'noop', verbFields }, or +// { kind: 'commit', ledgerOps, verbFields }. +function executeMirroredWrite(opts) { + const { state, root, tool, operationId, expectedStateRev, expectedStateGen, semanticArgs, prepare } = opts; + const argsHash = bindingHash(tool, semanticArgs, expectedStateRev, expectedStateGen); + if (!fs.existsSync(state.statePath(root))) return { kind: 'stateMissing' }; + + let outcome = null; + try { + state.withMirroredMutation(root, { action: tool, door: 'mcp', tool, operationId, argsHash }, (s, ledger) => { + const ring = Array.isArray(s.operations) ? s.operations : null; + const hit = ring ? ring.find((entry) => entry && entry.id === operationId) : undefined; + if (hit) { + outcome = hit.argsHash === argsHash + ? { kind: 'replayed', result: clone(hit.result) } + : { kind: 'conflict' }; + return null; + } + if (String(s.gen || '') !== expectedStateGen) { + outcome = { kind: 'staleGen', actualStateGen: String(s.gen || '') || null }; + return null; + } + const rev = revOf(s); + if (rev !== expectedStateRev) { + outcome = { kind: 'staleRev', actualStateRev: rev }; + return null; + } + const mintId = (prefix, role) => { + const id = deriveId(prefix, expectedStateGen, tool, argsHash, role); + if (recordIdExists(s, id)) { + const e = new Error(`derived id ${id} already names a record`); + e.code = 'ERATCHETIDCONFLICT'; + throw e; + } + return id; + }; + const prep = prepare(s, ledger, mintId); + if (!prep || prep.kind === 'noop') { + outcome = { kind: 'noop', result: successResult(false, rev, prep && prep.verbFields) }; + return null; + } + const result = successResult(true, rev + 1, prep.verbFields); + const entry = { + id: operationId, + tool, + argsHash, + gen: expectedStateGen, + rev: rev + 1, + at: schemas.nowIso(), + result, + }; + if (JSON.stringify(entry).length > RECEIPT_ENTRY_CAP) { + const e = new Error(`operation receipt exceeds ${RECEIPT_ENTRY_CAP} bytes`); + e.code = 'ERATCHETRECEIPTCAP'; + throw e; + } + if (!Array.isArray(s.operations)) s.operations = []; + s.operations.push(entry); + while (s.operations.length > OPERATIONS_CAP) s.operations.shift(); + outcome = { kind: 'committed', result }; + return { kind: 'commit', ledgerOps: prep.ledgerOps, result }; + }); + } catch (error) { + const kind = error && CODED_OUTCOMES[error.code]; + if (kind) return { kind }; + throw error; + } + return outcome; +} + module.exports = { OPERATIONS_CAP, RECEIPT_ENTRY_CAP, @@ -237,4 +317,5 @@ module.exports = { deriveId, recordIdExists, executeWrite, + executeMirroredWrite, }; diff --git a/src/mcp/server.js b/src/mcp/server.js index 848ff48..c1e382a 100644 --- a/src/mcp/server.js +++ b/src/mcp/server.js @@ -41,6 +41,9 @@ // twice. See docs/superpowers/specs/2026-07-31-mcp-write-tools-design.md. // Step 4 traced by: claude-fable-5 +const crypto = require('crypto'); +const fs = require('fs'); + const handles = require('./handles'); const ops = require('./ops'); const prompts = require('./prompts'); @@ -104,6 +107,9 @@ const TOOL = Object.freeze({ // recreated out-of-band, and a generation the client did not observe is // a world it never decided against. stateGen: { type: 'string' }, + // Derived, never persisted: whether a 4b write-ahead intent occupies the + // slot. Open recovers under its lock first, so this is normally false. + pendingIntent: { type: 'boolean' }, resources: { type: 'object', properties: { @@ -115,7 +121,7 @@ const TOOL = Object.freeze({ additionalProperties: false, }, }, - required: ['workspaceHandle', 'repositoryId', 'worktreeId', 'stateRev', 'stateGen', 'resources'], + required: ['workspaceHandle', 'repositoryId', 'worktreeId', 'stateRev', 'stateGen', 'pendingIntent', 'resources'], additionalProperties: false, }, annotations: { @@ -306,6 +312,8 @@ const WRITE_ERROR_BRANCH = Object.freeze({ 'ClosureBlocked', 'HumanAuthorityRequired', 'RetractRefused', + 'AttachmentAmbiguous', + 'MirrorUnrecoverable', 'WriteFailed', ], }, @@ -672,6 +680,114 @@ const SCORE_APERTURE_TOOL = Object.freeze({ }, }); +const DEFECT_ADD_USAGE = + 'defect.add requires exactly: workspaceHandle, expectedStateRev, expectedStateGen, operationId, item (an object)'; + +const DEFECT_ADD_TOOL = Object.freeze({ + name: 'defect.add', + title: 'Record a defect with its ledger mirror', + description: + 'Record one defect on an opened workspace — the first cross-file verb: the state record and its QA-ledger mirror commit behind one write-ahead intent, so a server death between the two files is a recoverable lag, never a permanent disagreement. A repeat of an open finding dedups as a no-op; a worse repeat escalates the severity in place. Terminal birth statuses refuse at the boundary; waivers stay CLI acts. With several live artifacts, item.artifact must name the one this defect attacks. CAS-bound like every write.', + inputSchema: { + type: 'object', + properties: Object.assign({}, WRITE_ENVELOPE_PROPS, { + item: { + type: 'object', + description: 'The defect payload ({severity, summary, artifact, feature}). Severity defaults to medium; unknown severities coerce to medium.', + }, + }), + required: [...WRITE_ENVELOPE_KEYS, 'item'], + additionalProperties: false, + }, + outputSchema: { + oneOf: [ + writeSuccessBranch({ + defectId: { type: 'string' }, + severity: { type: 'string', enum: [...schemas.SEVERITIES] }, + action: { type: 'string', enum: ['created', 'escalated', 'deduped'] }, + artifact: { type: ['string', 'null'] }, + attachedBy: { type: 'string' }, + ledgerId: { type: ['string', 'null'] }, + }), + WRITE_ERROR_BRANCH, + ], + }, + // An escalation overwrites the recorded severity in place — not additive. + annotations: WRITE_DESTRUCTIVE, +}); + +// The three wire transitions share one descriptor factory: same envelope, same +// success projection (defectId, status, ledgerId — the mirror the operation +// kept truthful), same destructive annotation. defect.waive is deliberately +// NOT built here: waivers are human risk acceptance with no MCP spelling. +function defectTransitionTool(name, title, description, semanticProps, required, statusConst) { + return Object.freeze({ + name, + title, + description, + inputSchema: { + type: 'object', + properties: Object.assign({}, WRITE_ENVELOPE_PROPS, semanticProps), + required: [...WRITE_ENVELOPE_KEYS, ...required], + additionalProperties: false, + }, + outputSchema: { + oneOf: [ + writeSuccessBranch({ + defectId: { type: 'string' }, + status: { const: statusConst }, + ledgerId: { type: ['string', 'null'] }, + }), + WRITE_ERROR_BRANCH, + ], + }, + annotations: WRITE_DESTRUCTIVE, + }); +} + +const DEFECT_RESOLVE_USAGE = + 'defect.resolve requires exactly: workspaceHandle, expectedStateRev, expectedStateGen, operationId, id, evidence (non-empty)'; +const DEFECT_RESOLVE_TOOL = defectTransitionTool( + 'defect.resolve', + 'Resolve a defect with proof', + 'Mark one defect resolved on an opened workspace — no proof, no resolve. The state transition and its ledger mirror commit behind one write-ahead intent; an exact repeat is a no-op, and a repeat with different proof refuses rather than silently replacing the original. CAS-bound like every write.', + { + id: { type: 'string', minLength: 1 }, + evidence: { type: 'string', minLength: 1, description: 'Proof it is actually fixed.' }, + }, + ['id', 'evidence'], + 'resolved' +); + +const DEFECT_REOPEN_USAGE = + 'defect.reopen requires exactly: workspaceHandle, expectedStateRev, expectedStateGen, operationId, id, reason (non-empty)'; +const DEFECT_REOPEN_TOOL = defectTransitionTool( + 'defect.reopen', + 'Reopen a defect that is not actually fixed', + 'Reopen one defect on an opened workspace with the reason it is not actually fixed. Mirrored behind one write-ahead intent; exact repeats are no-ops. CAS-bound like every write.', + { + id: { type: 'string', minLength: 1 }, + reason: { type: 'string', minLength: 1, description: 'Why it is not actually fixed.' }, + }, + ['id', 'reason'], + 'reopened' +); + +const DEFECT_SUPERSEDE_USAGE = + 'defect.supersede requires exactly: workspaceHandle, expectedStateRev, expectedStateGen, operationId, id, by (non-empty), optionally reason (non-empty)'; +const DEFECT_SUPERSEDE_TOOL = defectTransitionTool( + 'defect.supersede', + 'Supersede a defect with its replacement', + 'Mark one defect superseded on an opened workspace, naming the artifact or defect that replaced it. Mirrored behind one write-ahead intent; exact repeats are no-ops. CAS-bound like every write.', + { + id: { type: 'string', minLength: 1 }, + by: { type: 'string', minLength: 1, description: 'The artifact or defect that replaced it.' }, + reason: { type: 'string', minLength: 1, description: 'Optional context for the supersession.' }, + }, + ['id', 'by'], + 'superseded' +); + // The one sentence each refusal speaks. A table, so the funnel test can assert // every wire sentence against this allowlist — no verb-specific catch can leak // a path, an errno, or a store location. @@ -686,6 +802,9 @@ const WRITE_REFUSALS = Object.freeze({ ClosureBlocked: 'artifact closure is blocked — bound proof, open defects, holes, or a damaged proof record stand in the way; the confidence read names each blocker', HumanAuthorityRequired: 'this closure needs named human authorization (record-scope proof or waived holes) — it has no wire spelling; run it from the CLI', RetractRefused: 'retraction refused — a probe exit states its outcome (reason starts "disposed:" or "promoted:"), and a promotion names a recorded non-probe replacement', + // The 4b literals are pinned by the ratified spec, word for word. + AttachmentAmbiguous: 'Several live artifacts could own this defect; provide item.artifact explicitly.', + MirrorUnrecoverable: 'The defect mirror cannot be read or recovered safely; run ratchet doctor and repair the reported condition before retrying.', WriteFailed: 'workspace write could not be completed', }); @@ -757,6 +876,33 @@ function runWrite(record, tool, args, semanticArgs, apply, alsoLockFile) { } catch (error) { return safeWriteError(error); } + return writeOutcome(outcome, args); +} + +// The 4b variant: same boundary, same outcome mapping, but the execution runs +// the cross-file protocol. A post-decision mirror failure throws out of the +// executor and lands in safeWriteError — the retryable WriteFailed — because +// no success is emitted until the mirror and the clear have both landed. +function runMirroredWrite(record, tool, args, semanticArgs, prepare) { + let outcome; + try { + outcome = ops.executeMirroredWrite({ + state, + root: record.root, + tool, + operationId: args.operationId, + expectedStateRev: args.expectedStateRev, + expectedStateGen: args.expectedStateGen, + semanticArgs, + prepare, + }); + } catch (error) { + return safeWriteError(error); + } + return writeOutcome(outcome, args); +} + +function writeOutcome(outcome, args) { switch (outcome.kind) { case 'replayed': return toolResult(Object.assign(outcome.result, { replayed: true })); @@ -789,6 +935,10 @@ function runWrite(record, tool, args, semanticArgs, apply, alsoLockFile) { return writeRefusal('HumanAuthorityRequired'); case 'retractRefused': return writeRefusal('RetractRefused'); + case 'attachAmbiguous': + return writeRefusal('AttachmentAmbiguous'); + case 'mirror': + return writeRefusal('MirrorUnrecoverable'); default: return writeRefusal('WriteFailed'); } @@ -848,12 +998,53 @@ function safeOpenError(error) { if (error && error.code === 'ERATCHETGIT') { return 'workspace must be an accessible Git working tree directory'; } + if (error && error.code === 'ERATCHETMIRROR') { + // Open recovers the 4b intent slot before issuing a handle; a slot strict + // recovery cannot prove legal is an operator condition, same voice as the + // write tools' allowlisted sentence. + return 'The defect mirror cannot be read or recovered safely; run ratchet doctor and repair the reported condition before retrying.'; + } if (error && error.code && String(error.code).startsWith('ERATCHETHANDLE')) { return 'workspace authority could not be issued'; } return 'workspace state could not be opened'; } +// The 4b pendingIntent sample: the SHA-256 of readable raw intent bytes, an +// occupied-unreadable sentinel, or absent — never a bare existence bit, so two +// different slots never sample equal. Non-repairing by construction: one read, +// no lock, no store bytes created. +function intentToken(root) { + try { + return `sha256:${crypto.createHash('sha256').update(fs.readFileSync(state.intentPath(root))).digest('hex')}`; + } catch (e) { + return e && e.code === 'ENOENT' ? 'absent' : 'occupied-unreadable'; + } +} + +// Non-repairing revision peek for the sampler pair. readJson swallows a parse +// failure into null; 'rev:unknown' keeps that observation distinct from rev 0. +function revToken(root) { + const parsed = state.readJson(state.statePath(root)); + return parsed && Number.isInteger(parsed.rev) ? `rev:${parsed.rev}` : 'rev:unknown'; +} + +// Lock-free reads sample (state revision, intent token) around the assembly. +// A stable absent token plus a stable revision is a clean linearization point; +// anything unstable is conservatively reported as pendingIntent:true rather +// than letting a mixed WAL snapshot claim it saw a settled store. +function samplePendingIntent(root, assemble) { + for (let attempt = 0; attempt < 2; attempt++) { + const tokenBefore = intentToken(root); + const revBefore = revToken(root); + const value = assemble(); + if (intentToken(root) === tokenBefore && revToken(root) === revBefore) { + return { value, pendingIntent: tokenBefore !== 'absent' }; + } + } + return { value: assemble(), pendingIntent: true }; +} + function resourceUri(handle, name) { return `torque://workspace/${handle}/${name}`; } @@ -954,8 +1145,15 @@ function createServer(options) { // creates it when it is missing: without this, the first ledger // resource read of a fresh workspace wrote bytes, and "every read is // pure" was false on the one path a client hits first. - snapshot = state.loadState(found.root); - state.loadLedger(found.root); + // + // 4b: the snapshot is taken INSIDE the workspace lock, because the + // lock's post-acquire path is where pending-intent recovery lives — a + // healthy store used to be read lock-free here, which would have + // issued a handle over a mirror still owed its recovery. + state.withWorkspaceLock(found.root, 'workspace open', () => { + snapshot = state.loadState(found.root); + state.loadLedger(found.root); + }); } catch (error) { // Either record failing means no handle: authority over a workspace // whose canonical records could not be opened is authority to read @@ -1010,6 +1208,9 @@ function createServer(options) { // The same loaded snapshot as stateRev: a revision and a generation // read separately could describe two different records. stateGen: snapshot ? String(snapshot.gen || '') : '', + // Sampled after the locked recovery above — normally 'absent'; a later + // writer's slot is the state CAS contract's problem, not this field's. + pendingIntent: intentToken(record.root) !== 'absent', resources: uris, }; return { @@ -1244,6 +1445,79 @@ function createServer(options) { }); } + function defectAdd(arguments_) { + const args = writeArguments(arguments_, ['item'], DEFECT_ADD_USAGE); + const item = args.item; + if (!item || typeof item !== 'object' || Array.isArray(item)) throw rpc.rpcError(-32602, DEFECT_ADD_USAGE); + if (item.id !== undefined && !nonEmpty(item.id)) throw rpc.rpcError(-32602, DEFECT_ADD_USAGE); + // Terminal birth statuses refuse at the boundary, same rule as + // state.append; the shared core re-checks underneath. + const claimed = item.status == null ? '' : String(item.status).toLowerCase(); + if (claimed && schemas.DEFECT_TERMINAL_STATUSES.includes(claimed)) { + throw rpc.rpcError(-32602, + 'defect.add cannot birth a terminal status — resolve and supersede are transitions, and waivers stay CLI acts'); + } + const record = resolveHandle(args.workspaceHandle, 'write'); + return runMirroredWrite(record, 'defect.add', args, { item }, (s, ledger, mintId) => { + const prep = artifacts.prepareDefectAdd(record.root, s, ledger, item, mintId); + const rec = prep.record; + const verbFields = { + defectId: String(rec.id), + severity: rec.severity, + action: prep.action, + artifact: rec.artifact ? String(rec.artifact) : null, + attachedBy: String(rec.attachedBy || ''), + ledgerId: rec.ledgerId ? String(rec.ledgerId) : null, + }; + if (prep.kind === 'noop') return { kind: 'noop', verbFields }; + return { kind: 'commit', ledgerOps: prep.ledgerOps, verbFields }; + }); + } + + // One handler shape for the three wire transitions: boundary gates travel + // as schema, the shared core owns the meaning, and the mirror rides the + // same intent. waive has no handler here, by rule. + function runDefectTransition(args, toStatus, tool, meta, semanticArgs) { + const record = resolveHandle(args.workspaceHandle, 'write'); + return runMirroredWrite(record, tool, args, semanticArgs, (s, ledger, mintId) => { + const prep = artifacts.prepareDefectTransition(s, ledger, args.id, toStatus, meta, mintId); + const verbFields = { + defectId: String(prep.record.id), + status: toStatus, + ledgerId: prep.record.ledgerId ? String(prep.record.ledgerId) : null, + }; + if (prep.kind === 'noop') return { kind: 'noop', verbFields }; + return { kind: 'commit', ledgerOps: prep.ledgerOps, verbFields }; + }); + } + + function defectResolve(arguments_) { + const args = writeArguments(arguments_, ['id', 'evidence'], DEFECT_RESOLVE_USAGE); + if (!nonEmpty(args.id) || !nonEmpty(args.evidence)) throw rpc.rpcError(-32602, DEFECT_RESOLVE_USAGE); + return runDefectTransition(args, 'resolved', 'defect.resolve', + { evidence: args.evidence, note: `resolved: ${args.evidence}` }, + { id: args.id, evidence: args.evidence }); + } + + function defectReopen(arguments_) { + const args = writeArguments(arguments_, ['id', 'reason'], DEFECT_REOPEN_USAGE); + if (!nonEmpty(args.id) || !nonEmpty(args.reason)) throw rpc.rpcError(-32602, DEFECT_REOPEN_USAGE); + return runDefectTransition(args, 'reopened', 'defect.reopen', + { reason: args.reason, note: `reopened: ${args.reason}` }, + { id: args.id, reason: args.reason }); + } + + function defectSupersede(arguments_) { + const args = writeArguments(arguments_, ['id', 'by'], DEFECT_SUPERSEDE_USAGE, ['reason']); + if (!nonEmpty(args.id) || !nonEmpty(args.by)) throw rpc.rpcError(-32602, DEFECT_SUPERSEDE_USAGE); + if (args.reason !== undefined && !nonEmpty(args.reason)) throw rpc.rpcError(-32602, DEFECT_SUPERSEDE_USAGE); + // Absent and null are one meaning in the binding, same as artifact.retract. + const reason = args.reason === undefined ? null : args.reason; + return runDefectTransition(args, 'superseded', 'defect.supersede', + { by: args.by, reason: reason || '', note: `superseded by ${args.by}${reason ? `: ${reason}` : ''}` }, + { id: args.id, by: args.by, reason }); + } + function scoreAperture(arguments_) { const args = writeArguments(arguments_, [...APERTURE_DIMENSION_KEYS], SCORE_APERTURE_USAGE); const dims = {}; @@ -1283,6 +1557,10 @@ function createServer(options) { { descriptor: ARTIFACT_CLOSE_TOOL, run: artifactClose }, { descriptor: ARTIFACT_RETRACT_TOOL, run: artifactRetract }, { descriptor: SCORE_APERTURE_TOOL, run: scoreAperture }, + { descriptor: DEFECT_ADD_TOOL, run: defectAdd }, + { descriptor: DEFECT_RESOLVE_TOOL, run: defectResolve }, + { descriptor: DEFECT_REOPEN_TOOL, run: defectReopen }, + { descriptor: DEFECT_SUPERSEDE_TOOL, run: defectSupersede }, ] : []), ]; @@ -1301,10 +1579,15 @@ function createServer(options) { function readResource(params, context) { const available = resourceRecord(params.uri); - let value; - if (available.parsed.name === 'state') value = state.loadState(available.record.root); - else if (available.parsed.name === 'ledger') value = state.loadLedger(available.record.root); - else value = receipt.assemble(available.record.root); + // 4b: every resource states the WAL observation out loud. The flag is + // derived at read time and injected into the projection only — the disk + // bytes never gain it. Reads stay byte-pure: the sampler never repairs. + const sampled = samplePendingIntent(available.record.root, () => { + if (available.parsed.name === 'state') return state.loadState(available.record.root); + if (available.parsed.name === 'ledger') return state.loadLedger(available.record.root); + return receipt.assemble(available.record.root); + }); + const value = Object.assign({}, sampled.value, { pendingIntent: sampled.pendingIntent }); return withCache({ contents: [{ diff --git a/src/state.js b/src/state.js index db3a821..265d815 100644 --- a/src/state.js +++ b/src/state.js @@ -7,6 +7,7 @@ const crypto = require('crypto'); const { isDeepStrictEqual } = require('util'); const schemas = require('./schemas'); +const wal = require('./wal'); // --------------------------------------------------------------------------- // Location. State survives plugin updates when CLAUDE_PLUGIN_DATA is set. @@ -145,6 +146,11 @@ function ledgerPath(cwd) { return path.join(projectDir(cwd), 'ledger.json'); } +// The 4b write-ahead intent slot. One per store, beside the records it binds. +function intentPath(cwd) { + return path.join(projectDir(cwd), 'intent.json'); +} + function ensureDir(dir) { fs.mkdirSync(dir, { recursive: true }); } @@ -190,8 +196,31 @@ function writeFileAtomic(file, data, beforePublish) { } fs.closeSync(fd); fd = undefined; - if (beforePublish) beforePublish(); - fs.renameSync(tmp, file); + // Windows can refuse a rename over a destination some other process holds + // open for an instant (an indexer, a scanner, a reader mid-poll) — EPERM + // on a publish that would succeed 10ms later. Retry briefly, re-running + // the fence each attempt so a holding broken in the window still refuses; + // a persistent refusal still throws, it is never swallowed. + // A rename over a destination some other process holds open (an AV scan + // of the bytes the PREVIOUS publish just wrote is the measured culprit) + // refuses EPERM on Windows until the hold drops — for seconds when the + // machine is loaded. Fixed backoffs lost that race in practice, so this + // waits like acquireLock does: against a deadline, generous because the + // publish is rare and already inside the lock's own 15s patience. Git + // ships the same shaped loop for the same reason. A genuinely held + // destination still throws at the deadline; nothing is ever swallowed. + const deadline = Date.now() + envMs('RATCHET_PUBLISH_TIMEOUT_MS', 10000); + for (;;) { + try { + if (beforePublish) beforePublish(); + fs.renameSync(tmp, file); + break; + } catch (renameError) { + if (!renameError || (renameError.code !== 'EPERM' && renameError.code !== 'EACCES')) throw renameError; + if (Date.now() >= deadline) throw renameError; + sleepSync(50); + } + } } catch (e) { // A half-written temp file is residue, not a record. Name it, then drop it. if (fd !== undefined) { @@ -693,6 +722,109 @@ function assertLockOrder(what) { } } +// 4b: the recovery choke point. Every supported canonical writer reaches the +// store through one of the two lock APIs below, so recovery lives HERE — in +// the shared post-acquire path — not in each caller. A nested helper joins a +// scope that has already recovered; the re-entry guard keeps recovery's own +// strict reads and fenced publishes from recursing into a second recovery. +// An occupied slot recovery cannot prove legal throws ERATCHETMIRROR: no +// writer proceeds over a store whose mirror cannot be made truthful, and the +// refusal moves zero bytes (spec: 4b WAL design, ratified 2026-07-31). +let _recovering = false; + +function recoverPendingIntentLocked(cwd) { + if (_recovering) return; + const file = intentPath(cwd); + // One stat per acquisition is the whole fast-path cost. + if (!fs.existsSync(file)) return; + _recovering = true; + try { + wal.recover({ + intentFile: file, + statePath: statePath(cwd), + ledgerPath: ledgerPath(cwd), + publishLedger: (bytes) => { + try { + writeFileAtomic(ledgerPath(cwd), bytes, () => fenceForFile(ledgerPath(cwd))); + } catch (e) { + if (e && (e.code === 'EPERM' || e.code === 'EACCES' || e.code === 'EBUSY')) { + // Recovery could not win the mirror publish either (a scanner can + // hold a freshly replaced file past even the publish deadline). + // The slot is intact and the owed mirror unchanged — this is the + // SAME retryable condition as a post-decision failure, and it + // wears the same code so every caller says "re-run", not EPERM. + const err = new Error(`the mirror is still pending recovery — re-run the command: ${e.message}`); + err.code = 'ERATCHETMIRRORPENDING'; + throw err; + } + throw e; + } + }, + clearIntent: () => clearIntentFile(file), + validateMcpReceipt: validateMirrorReceipt, + }); + } finally { + _recovering = false; + } +} + +// The slot delete gets the same transient-refusal tolerance as the publish +// rename: a scanner holding intent.json for an instant must not fail an +// operation whose work is already done. A persistent refusal throws and the +// slot survives — recovery clears it later, which is exactly what it is for. +function clearIntentFile(file) { + const deadline = Date.now() + envMs('RATCHET_PUBLISH_TIMEOUT_MS', 10000); + for (;;) { + try { + fenceForFile(file); + fs.unlinkSync(file); + return; + } catch (e) { + if (!e || (e.code !== 'EPERM' && e.code !== 'EACCES' && e.code !== 'EBUSY')) throw e; + if (Date.now() >= deadline) throw e; + sleepSync(50); + } + } +} + +// An MCP state after-image must carry exactly the receipt its intent names — +// an after-hash match with a missing or contradicting receipt is a store this +// build cannot explain. CLI intents carry no receipt; the exact state hash is +// their evidence. +function validateMirrorReceipt(stateObj, intent) { + if (intent.door !== 'mcp') return true; + const ring = Array.isArray(stateObj.operations) ? stateObj.operations : []; + const hits = ring.filter((e) => e && e.id === intent.operationId); + if (hits.length !== 1) return false; + const hit = hits[0]; + return ( + hit.argsHash === intent.argsHash && + String(hit.gen || '') === intent.stateGen && + hit.rev === intent.targetStateRev && + Boolean(hit.result) && typeof hit.result === 'object' && hit.result.ok === true + ); +} + +// Read-only 4b diagnosis for doctor and tests: what recovery WOULD do with the +// slot, or the exact local reason it refuses. Never publishes, clears, backs +// up, or repairs — the lock-free read is safe because it moves nothing. +function diagnoseIntent(cwd) { + try { + const res = wal.recover({ + intentFile: intentPath(cwd), + statePath: statePath(cwd), + ledgerPath: ledgerPath(cwd), + publishLedger: () => {}, + clearIntent: () => {}, + validateMcpReceipt: validateMirrorReceipt, + }, { dryRun: true }); + return res.pending ? { pending: true, verdict: res.verdict } : { pending: false }; + } catch (e) { + if (e && e.code === 'ERATCHETMIRROR') return { pending: true, verdict: 'ambiguous', reason: e.message }; + throw e; + } +} + // Lock a store directory for the duration of fn. Use this for a public command // whose write is not a state.json revision (ledger upserts, the init/reset wipe) // — everything that revises state goes through withWorkspaceMutation below. @@ -708,6 +840,7 @@ function withWorkspaceLock(cwd, action, fn) { const handle = acquireLock(path.join(dir, LOCK_DIR_NAME), action); _scope = { dir, action, state: null, handle }; try { + recoverPendingIntentLocked(cwd); return fn(); } finally { _scope = null; @@ -1237,6 +1370,7 @@ function withWorkspaceMutation(cwd, opts, mutate) { const handle = acquireLock(path.join(dir, LOCK_DIR_NAME), action); _scope = { dir, action, state: null, handle }; try { + recoverPendingIntentLocked(cwd); // The second lock, if the verb asked for one, wraps everything below — // validation AND commit — and is released by withFileLock's own finally. return o.alsoLockFile @@ -1292,6 +1426,144 @@ function runMutation(cwd, o, action, mutate) { } } +// THE cross-file transaction boundary (4b). One operation writes state.json +// AND ledger.json behind one write-ahead intent: recover → prepare in memory → +// materialize exact post-image bytes → publish the intent create-exclusive → +// commit state (the decision) → publish the mirror → clear the slot. A death +// anywhere leaves one of the three legal hash pairs, and the next supported +// writer's recovery finishes or discards the work — never half-keeps it. +// +// `prepare(s, ledger)` mutates the transaction's state object like any verb +// and returns: +// null / { kind: 'skip' } — zero writes (refusal carried by the caller) +// { kind: 'noop', result } — zero writes, an idempotent answer +// { kind: 'commit', ledgerOps, result } — the full protocol +// Both records are STRICTLY loaded: present-but-unprovable bytes refuse +// (ERATCHETMIRROR) instead of meeting the repairing loaders — a WAL that +// hashes bytes cannot stand on a loader that rewrites them. +function withMirroredMutation(cwd, opts, prepare) { + const o = opts || {}; + const action = o.action || 'mirrored mutation'; + if (_scope) { + throw new Error( + `nested workspace mutation refused: "${action}" inside "${_scope.action}" — one public command is one ` + + 'transaction, and helpers mutate the open transaction instead of opening their own.' + ); + } + assertMayWrite(action); + const dir = projectDir(cwd); + assertLockOrder(action); + const handle = acquireLock(path.join(dir, LOCK_DIR_NAME), action); + _scope = { dir, action, state: null, handle }; + try { + recoverPendingIntentLocked(cwd); + return runMirrored(cwd, o, action, prepare); + } finally { + _scope = null; + releaseLock(handle); + } +} + +// Strict in-scope read for the WAL path: exact bytes plus their parse, with an +// ENOENT escape the caller may answer by creating the record (a first CLI +// defect on a fresh store must still work). Never a backup, never a repair. +function readForMirror(file, what) { + let bytes; + try { + bytes = fs.readFileSync(file); + } catch (e) { + if (e && e.code === 'ENOENT') return null; + const err = new Error(`${what} exists but cannot be read strictly (${e && e.code ? e.code : 'unknown'})`); + err.code = 'ERATCHETMIRROR'; + throw err; + } + try { + const parsed = JSON.parse(bytes.toString('utf8')); + if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) throw new Error('not a record'); + return { bytes, parsed }; + } catch (_e) { + const err = new Error(`${what} is present but not a readable record — repair it before a mirrored write`); + err.code = 'ERATCHETMIRROR'; + throw err; + } +} + +function runMirrored(cwd, o, action, prepare) { + // Absent records are created through the ordinary boundaries (that is a + // plain single-file write), then re-read strictly so the hashes cover the + // exact bytes on disk. + if (!fs.existsSync(statePath(cwd))) loadState(cwd); + if (!fs.existsSync(ledgerPath(cwd))) loadLedger(cwd); + const stateRead = readForMirror(statePath(cwd), 'state record'); + const ledgerRead = readForMirror(ledgerPath(cwd), 'ledger record'); + if (!stateRead || !ledgerRead) { + const err = new Error('canonical record vanished inside the transaction'); + err.code = 'ERATCHETMIRROR'; + throw err; + } + const s = rememberBase(cwd, stateRead.parsed); + const baseRev = revOf(s); + _scope.state = s; + const prep = prepare(s, ledgerRead.parsed); + _scope.state = null; + if (!prep || prep.kind === 'skip') return { committed: false, rev: baseRev, state: s, result: prep && prep.result }; + if (prep.kind === 'noop') return { committed: false, rev: baseRev, state: s, result: prep.result }; + + // MATERIALIZE once: every stamp is final before the intent publishes, and + // the hashes cover the exact bytes both publishes will write. + assertStillOwner(cwd, `state rev ${baseRev + 1}`); + const now = schemas.nowIso(); + s.updatedAt = now; + s.rev = baseRev + 1; + const stateAfterBytes = wal.serializeRecord(s); + const ledgerAfter = wal.applyLedgerOps(ledgerRead.parsed, prep.ledgerOps, now); + const ledgerAfterBytes = wal.serializeRecord(ledgerAfter); + const intent = { + version: 1, + door: o.door, + operationId: o.operationId, + tool: o.tool || action, + argsHash: o.argsHash, + stateGen: String(s.gen || '') || '(none)', + baseStateRev: baseRev, + targetStateRev: baseRev + 1, + stateBeforeHash: wal.hashBytes(stateRead.bytes), + stateAfterHash: wal.hashBytes(stateAfterBytes), + ledgerBeforeHash: wal.hashBytes(ledgerRead.bytes), + ledgerAfterHash: wal.hashBytes(ledgerAfterBytes), + ledgerUpdatedAt: now, + ledgerOps: prep.ledgerOps, + at: now, + }; + // Self-check: the slot we publish must be one our own recovery accepts — + // cap included. Failing closed here costs nothing; failing open costs a + // store nobody can recover. + wal.parseIntent(Buffer.from(wal.serializeRecord(intent), 'utf8')); + if (!createJsonExclusive(intentPath(cwd), intent)) { + const err = new Error('the intent slot is occupied — recovery should have resolved it; refusing to overwrite'); + err.code = 'ERATCHETMIRROR'; + throw err; + } + + // THE decision: the exact hashed state bytes, one rename. + writeFileAtomic(statePath(cwd), stateAfterBytes, () => fenceForFile(statePath(cwd))); + rememberBase(cwd, s); + + // Post-decision: a failure here leaves the slot for the next writer's + // recovery. The operation HAS happened; only the answer must say "pending". + try { + writeFileAtomic(ledgerPath(cwd), ledgerAfterBytes, () => fenceForFile(ledgerPath(cwd))); + clearIntentFile(intentPath(cwd)); + } catch (e) { + const err = new Error( + `the state change committed (rev ${s.rev}) but the mirror is pending recovery — re-run the command: ${e && e.message}` + ); + err.code = 'ERATCHETMIRRORPENDING'; + throw err; + } + return { committed: true, rev: s.rev, state: s, result: prep.result }; +} + function loadLedger(cwd) { const existing = readJsonResilient(ledgerPath(cwd)); if (existing) return existing; @@ -1340,6 +1612,7 @@ module.exports = { projectDir, statePath, ledgerPath, + intentPath, ensureDir, readJson, writeJson, @@ -1350,7 +1623,9 @@ module.exports = { saveState, withWorkspaceLock, withWorkspaceMutation, + withMirroredMutation, withFileLock, + diagnoseIntent, loadLedger, saveLedger, makeId, diff --git a/src/wal.js b/src/wal.js new file mode 100644 index 0000000..50dce58 Binary files /dev/null and b/src/wal.js differ diff --git a/test/fixtures/mcp-tools-list.json b/test/fixtures/mcp-tools-list.json index bf8b538..b44a6a1 100644 --- a/test/fixtures/mcp-tools-list.json +++ b/test/fixtures/mcp-tools-list.json @@ -34,6 +34,9 @@ "stateGen": { "type": "string" }, + "pendingIntent": { + "type": "boolean" + }, "resources": { "type": "object", "properties": { @@ -61,6 +64,7 @@ "worktreeId", "stateRev", "stateGen", + "pendingIntent", "resources" ], "additionalProperties": false diff --git a/test/mcp-wal.test.js b/test/mcp-wal.test.js new file mode 100644 index 0000000..3cf8a83 --- /dev/null +++ b/test/mcp-wal.test.js @@ -0,0 +1,899 @@ +'use strict'; + +// Torque step 4b.1: the write-ahead intent slot on the defect.add canary. +// Run: node test/mcp-wal.test.js +// +// What this suite exists to prove, in the ratified spec's words: one operation +// writes two canonical files behind one intent, a process death anywhere lands +// on one of three legal hash pairs, recovery finishes or discards the work — +// byte-exactly, proven against the hashes the intent recorded — and anything +// recovery cannot prove preserves every byte and refuses loudly. The crash +// matrix uses REAL child processes dying at the actual rename/link/unlink +// calls, on both the normal path and inside recovery itself. +// +// Traced by: claude-fable-5 + +const assert = require('assert'); +const childProcess = require('child_process'); +const crypto = require('crypto'); +const fs = require('fs'); +const os = require('os'); +const path = require('path'); + +const tmp = fs.realpathSync.native( + fs.mkdtempSync(path.join(os.tmpdir(), 'ratchet-mcp-wal-test-')) +); +process.env.RATCHET_DATA_DIR = path.join(tmp, 'state-store'); +process.env.RATCHET_EVOLVE_LOG = path.join(tmp, 'evolve-log.jsonl'); + +const wal = require('../src/wal'); +const state = require('../src/state'); +const artifacts = require('../src/artifacts'); +const mcp = require('../src/mcp/server'); + +const META = 'io.modelcontextprotocol/'; +const MODERN = '2026-07-28'; + +let passed = 0; +const failures = []; +function ok(name, fn) { + try { + fn(); + passed++; + process.stdout.write(` ok ${name}\n`); + } catch (e) { + failures.push(name); + process.stdout.write(` FAIL ${name}\n ${e && e.message ? e.message : e}\n`); + } +} + +let fixtureNumber = 0; +function fixture(label) { + const dir = path.join(tmp, `${label}-${fixtureNumber++}`); + fs.mkdirSync(dir, { recursive: true }); + return fs.realpathSync.native(dir); +} + +function cleanGitEnv() { + const env = Object.assign({}, process.env); + for (const key of Object.keys(env)) { + if (key.toUpperCase().startsWith('GIT_')) delete env[key]; + } + return env; +} + +function git(cwd, args) { + return childProcess.execFileSync('git', args, { + cwd, encoding: 'utf8', env: cleanGitEnv(), stdio: ['ignore', 'pipe', 'pipe'], windowsHide: true, + }); +} + +function initRepo(label) { + const dir = fixture(label); + git(dir, ['init', '--quiet']); + return dir; +} + +function initStore(repo) { + state.initProject(repo); + state.loadLedger(repo); +} + +function readIntent(repo) { + return JSON.parse(fs.readFileSync(state.intentPath(repo), 'utf8')); +} + +function bytesOf(file) { + return fs.readFileSync(file); +} + +function hashOf(file) { + return wal.hashBytes(bytesOf(file)); +} + +// Canonical bytes only. A process killed mid-transaction leaves its lock +// owner card and (before the intent publishes) a named .tmp- scratch file — +// both inert, both self-describing, both explicitly outside the recovery +// claim, which covers the two records and the slot. +function storeSnapshot(repo) { + const dir = state.projectDir(repo); + const out = {}; + if (!fs.existsSync(dir)) return out; + const walk = (d, rel) => { + for (const name of fs.readdirSync(d)) { + if (name === '.lock' || name.includes('.tmp-')) continue; + const full = path.join(d, name); + const key = rel ? `${rel}/${name}` : name; + const stat = fs.lstatSync(full); + if (stat.isDirectory()) walk(full, key); + else out[key] = fs.readFileSync(full).toString('hex'); + } + }; + walk(dir, ''); + return out; +} + +function readState(repo) { + return JSON.parse(fs.readFileSync(state.statePath(repo), 'utf8')); +} + +function readLedger(repo) { + return JSON.parse(fs.readFileSync(state.ledgerPath(repo), 'utf8')); +} + +// The spec's own degraded outcome, honored the way a user would. On this +// host, an external hold on a freshly replaced mirror can outlive even the +// publish deadline (measured: src movable, dest opens r+, the replace refused +// for seconds) — the operation then surfaces ERATCHETMIRRORPENDING and says +// "re-run the command". The harness does exactly that, a bounded number of +// times with a breather between. What the assertions pin is CONVERGENCE — +// recovery completes the mirror and the verb no-ops or answers — never a +// silent swallow of a different error. +function settled(fn) { + for (let attempt = 0; ; attempt++) { + try { + return fn(); + } catch (e) { + if (!e || e.code !== 'ERATCHETMIRRORPENDING' || attempt >= 2) throw e; + sleep(500); + } + } +} + +function sleep(ms) { + Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms); +} + +// Trigger the recovery choke point through a supported writer that changes +// nothing itself: an empty locked section — with the same settled patience, +// because recovery's own mirror publish can meet the same external hold. +function triggerRecovery(repo) { + settled(() => state.withWorkspaceLock(repo, 'wal-test recovery probe', () => {})); +} + +// A store with one committed defect and a crafted, internally-consistent +// intent in the requested phase. Crafting (rather than crashing) gives the +// ambiguity tests a slot whose every field is under test control; the crash +// matrix below produces the real thing. +function craftedSlot(repo, phase) { + initStore(repo); + const stateBytes = bytesOf(state.statePath(repo)); + const ledgerBytes = bytesOf(state.ledgerPath(repo)); + const ledgerObj = JSON.parse(ledgerBytes.toString('utf8')); + const mirror = { + id: 'ldef-crafted', at: '2026-07-31T00:00:00.000Z', feature: '', severity: 'high', + summary: 'crafted', status: 'open', foundAt: '2026-07-31T00:00:00.000Z', + }; + const ops = [{ collection: 'defects', id: 'ldef-crafted', mode: 'insert', after: mirror }]; + const now = '2026-07-31T00:00:00.000Z'; + const stateObj = JSON.parse(stateBytes.toString('utf8')); + stateObj.defects.push({ id: 'def-crafted', at: now, severity: 'high', summary: 'crafted', status: 'open', artifact: '', attachedBy: 'none', artifactRev: null, artifactHash: '', ledgerId: 'ldef-crafted' }); + stateObj.dirty = true; + stateObj.rev = (Number.isInteger(stateObj.rev) ? stateObj.rev : 0) + 1; + stateObj.updatedAt = now; + const stateAfterBytes = wal.serializeRecord(stateObj); + const ledgerAfterBytes = wal.serializeRecord(wal.applyLedgerOps(ledgerObj, ops, now)); + const intent = { + version: 1, door: 'cli', operationId: 'wal-crafted-operation', tool: 'defect add', + argsHash: wal.hashBytes(Buffer.from('crafted', 'utf8')), + stateGen: String(stateObj.gen || '') || '(none)', + baseStateRev: stateObj.rev - 1, targetStateRev: stateObj.rev, + stateBeforeHash: wal.hashBytes(stateBytes), + stateAfterHash: wal.hashBytes(stateAfterBytes), + ledgerBeforeHash: wal.hashBytes(ledgerBytes), + ledgerAfterHash: wal.hashBytes(ledgerAfterBytes), + ledgerUpdatedAt: now, ledgerOps: ops, at: now, + }; + if (phase === 'after-state' || phase === 'after-ledger') { + fs.writeFileSync(state.statePath(repo), stateAfterBytes); + } + if (phase === 'after-ledger') { + fs.writeFileSync(state.ledgerPath(repo), ledgerAfterBytes); + } + fs.writeFileSync(state.intentPath(repo), wal.serializeRecord(intent)); + return { intent, stateAfterBytes, ledgerAfterBytes }; +} + +// --------------------------------------------------------------------------- +// Unit: the strict parser and the op applier. +// --------------------------------------------------------------------------- + +function validIntent(over) { + const sha = `sha256:${'0'.repeat(64)}`; + return Object.assign({ + version: 1, door: 'cli', operationId: 'op-1234567890', tool: 'defect add', + argsHash: sha, stateGen: 'gen-x', baseStateRev: 3, targetStateRev: 4, + stateBeforeHash: sha, stateAfterHash: sha, ledgerBeforeHash: sha, ledgerAfterHash: sha, + ledgerUpdatedAt: 't', at: 't', + ledgerOps: [{ collection: 'defects', id: 'ldef-1', mode: 'insert', after: { id: 'ldef-1' } }], + }, over || {}); +} + +function parses(intent) { + return wal.parseIntent(Buffer.from(wal.serializeRecord(intent), 'utf8')); +} + +ok('U1 the version-1 parser accepts exactly the documented shape and nothing else', () => { + assert.ok(parses(validIntent())); + const rejects = [ + validIntent({ version: 2 }), + validIntent({ door: 'ssh' }), + validIntent({ tool: 'ledger.update' }), + validIntent({ operationId: '' }), + validIntent({ argsHash: 'sha256:short' }), + validIntent({ targetStateRev: 5 }), + validIntent({ baseStateRev: -1 }), + validIntent({ stateAfterHash: 'nope' }), + validIntent({ ledgerOps: [] }), + validIntent({ ledgerOps: [{ collection: 'features', id: 'x', mode: 'insert', after: { id: 'x' } }] }), + validIntent({ ledgerOps: [{ collection: 'defects', id: 'x', mode: 'merge', after: { id: 'x' } }] }), + validIntent({ ledgerOps: [{ collection: 'defects', id: 'x', mode: 'insert', after: { id: 'y' } }] }), + validIntent({ + ledgerOps: [ + { collection: 'defects', id: 'x', mode: 'insert', after: { id: 'x' } }, + { collection: 'defects', id: 'x', mode: 'replace', after: { id: 'x' } }, + ], + }), + Object.assign(validIntent(), { extra: true }), + ]; + for (const bad of rejects) { + assert.throws(() => parses(bad), (e) => e.code === 'ERATCHETMIRROR', JSON.stringify(bad).slice(0, 120)); + } + const missing = validIntent(); + delete missing.ledgerUpdatedAt; + assert.throws(() => parses(missing), (e) => e.code === 'ERATCHETMIRROR'); + assert.throws(() => wal.parseIntent(Buffer.from('not json', 'utf8')), (e) => e.code === 'ERATCHETMIRROR'); + assert.throws(() => wal.parseIntent(Buffer.alloc(wal.INTENT_CAP + 1)), (e) => e.code === 'ERATCHETMIRROR'); +}); + +ok('U2 ledger ops apply exactly: insert needs absence, replace needs exactly one', () => { + const ledger = { defects: [{ id: 'a', severity: 'low' }], features: [], tests: [], updatedAt: 'old' }; + const out = wal.applyLedgerOps(ledger, [ + { collection: 'defects', id: 'b', mode: 'insert', after: { id: 'b', severity: 'high' } }, + { collection: 'defects', id: 'a', mode: 'replace', after: { id: 'a', severity: 'critical' } }, + ], 'new-time'); + assert.deepStrictEqual(out.defects.map((d) => [d.id, d.severity]), [['a', 'critical'], ['b', 'high']]); + assert.strictEqual(out.updatedAt, 'new-time'); + assert.deepStrictEqual(ledger.defects.map((d) => d.severity), ['low'], 'the before-image is never mutated'); + assert.throws(() => wal.applyLedgerOps(ledger, [{ collection: 'defects', id: 'a', mode: 'insert', after: { id: 'a' } }], 't'), + (e) => e.code === 'ERATCHETMIRROR'); + assert.throws(() => wal.applyLedgerOps(ledger, [{ collection: 'defects', id: 'zz', mode: 'replace', after: { id: 'zz' } }], 't'), + (e) => e.code === 'ERATCHETMIRROR'); +}); + +ok('U3 the canonical publish retries a transient rename refusal; persistent refusals still throw', () => { + const dir = fixture('u3'); + const file = path.join(dir, 'target.json'); + const orig = fs.renameSync; + let denials = 2; + fs.renameSync = (a, b) => { + if (path.basename(String(b)) === 'target.json' && denials > 0) { + denials--; + const e = new Error('injected transient denial'); + e.code = 'EPERM'; + throw e; + } + return orig(a, b); + }; + try { + state.writeFileAtomic(file, 'published\n'); + } finally { + fs.renameSync = orig; + } + assert.strictEqual(fs.readFileSync(file, 'utf8'), 'published\n', + 'a scanner holding the destination for an instant does not fail the publish'); + fs.renameSync = (a, b) => { + if (path.basename(String(b)) === 'never.json') { + const e = new Error('denied for good'); + e.code = 'EPERM'; + throw e; + } + return orig(a, b); + }; + process.env.RATCHET_PUBLISH_TIMEOUT_MS = '150'; + try { + assert.throws(() => state.writeFileAtomic(path.join(dir, 'never.json'), 'x\n'), /denied for good/, + 'the deadline never becomes a swallow'); + } finally { + fs.renameSync = orig; + delete process.env.RATCHET_PUBLISH_TIMEOUT_MS; + } +}); + +// --------------------------------------------------------------------------- +// The three-state machine and its refusal to guess. +// --------------------------------------------------------------------------- + +ok('R1 a slot whose state decision never published is discarded byte-purely', () => { + const repo = fixture('r1'); + craftedSlot(repo, 'before'); + const stateHash = hashOf(state.statePath(repo)); + const ledgerHash = hashOf(state.ledgerPath(repo)); + assert.deepStrictEqual(state.diagnoseIntent(repo), { pending: true, verdict: 'discarded' }); + triggerRecovery(repo); + assert.ok(!fs.existsSync(state.intentPath(repo)), 'the slot is cleared'); + assert.strictEqual(hashOf(state.statePath(repo)), stateHash, 'state bytes untouched'); + assert.strictEqual(hashOf(state.ledgerPath(repo)), ledgerHash, 'ledger bytes untouched'); +}); + +ok('R2 a state-decided slot completes its mirror to the exact recorded bytes', () => { + const repo = fixture('r2'); + const { intent } = craftedSlot(repo, 'after-state'); + assert.deepStrictEqual(state.diagnoseIntent(repo), { pending: true, verdict: 'completed' }); + triggerRecovery(repo); + assert.ok(!fs.existsSync(state.intentPath(repo))); + assert.strictEqual(hashOf(state.statePath(repo)), intent.stateAfterHash); + assert.strictEqual(hashOf(state.ledgerPath(repo)), intent.ledgerAfterHash, + 'the recovered mirror is byte-identical to the bytes the intent proved'); + const disk = readState(repo); + const mirror = readLedger(repo).defects.find((d) => d.id === disk.defects[0].ledgerId); + assert.strictEqual(mirror.severity, disk.defects[0].severity, 'mirror and state agree'); +}); + +ok('R3 a slot whose mirror already landed only clears', () => { + const repo = fixture('r3'); + craftedSlot(repo, 'after-ledger'); + const before = storeSnapshot(repo); + assert.deepStrictEqual(state.diagnoseIntent(repo), { pending: true, verdict: 'cleared' }); + triggerRecovery(repo); + assert.ok(!fs.existsSync(state.intentPath(repo))); + delete before['intent.json']; + assert.deepStrictEqual(storeSnapshot(repo), before, 'nothing but the slot moved'); +}); + +ok('R4 every unprovable observation preserves every byte and refuses by name', () => { + const variants = [ + ['tampered state bytes', (repo) => { + craftedSlot(repo, 'after-state'); + const s = readState(repo); + s.objective = 'tampered out-of-band'; + fs.writeFileSync(state.statePath(repo), wal.serializeRecord(s)); + }], + ['corrupt intent json', (repo) => { + craftedSlot(repo, 'before'); + fs.writeFileSync(state.intentPath(repo), '{not json'); + }], + ['unknown intent version', (repo) => { + const { intent } = craftedSlot(repo, 'before'); + intent.version = 9; + fs.writeFileSync(state.intentPath(repo), wal.serializeRecord(intent)); + }], + ['mcp door without its receipt', (repo) => { + const { intent } = craftedSlot(repo, 'after-state'); + intent.door = 'mcp'; + fs.writeFileSync(state.intentPath(repo), wal.serializeRecord(intent)); + }], + ['reconstruction cannot reproduce the after-hash', (repo) => { + const { intent } = craftedSlot(repo, 'after-state'); + intent.ledgerAfterHash = `sha256:${'f'.repeat(64)}`; + fs.writeFileSync(state.intentPath(repo), wal.serializeRecord(intent)); + }], + ]; + for (const [name, arm] of variants) { + const repo = fixture('r4'); + arm(repo); + const before = storeSnapshot(repo); + assert.strictEqual(state.diagnoseIntent(repo).verdict, 'ambiguous', name); + assert.throws(() => triggerRecovery(repo), (e) => e.code === 'ERATCHETMIRROR', name); + assert.throws(() => artifacts.addDefect(repo, { summary: 'blocked' }), (e) => e.code === 'ERATCHETMIRROR', + `${name}: the mirrored writer refuses too`); + assert.deepStrictEqual(storeSnapshot(repo), before, `${name}: every byte preserved, slot included`); + } +}); + +ok('R5 the choke point covers the supported writers, not just the mirrored one', () => { + // Each supported door, given a discardable slot, resolves it before working. + const doors = [ + ['withWorkspaceMutation', (repo) => state.withWorkspaceMutation(repo, { action: 'probe' }, () => {})], + ['saveLedger', (repo) => state.saveLedger(repo, state.loadLedger(repo))], + ['initProject', (repo) => state.initProject(repo)], + ]; + for (const [name, act] of doors) { + const repo = fixture('r5'); + craftedSlot(repo, 'before'); + act(repo); + assert.ok(!fs.existsSync(state.intentPath(repo)), `${name} recovered the slot on its way in`); + } +}); + +// --------------------------------------------------------------------------- +// The CLI door: defect add through the WAL. +// --------------------------------------------------------------------------- + +ok('D1 a CLI defect add commits state, mirror, and back-link in one operation, slot cleared', () => { + const repo = fixture('d1'); + initStore(repo); + const res = settled(() => artifacts.addDefect(repo, { severity: 'high', summary: 'wal canary' })); + assert.ok(!fs.existsSync(state.intentPath(repo)), 'the slot does not outlive the operation'); + const disk = readState(repo); + const defect = disk.defects[0]; + assert.strictEqual(defect.id, res.state.id); + assert.ok(defect.ledgerId, 'the back-link rides the same post-image'); + const mirror = readLedger(repo).defects.find((d) => d.id === defect.ledgerId); + assert.ok(mirror, 'the mirror landed'); + assert.deepStrictEqual( + { severity: mirror.severity, summary: mirror.summary, status: mirror.status }, + { severity: 'high', summary: 'wal canary', status: 'open' }, + 'mirror and state agree' + ); + // A settled retry answers with the dedup shape (ledger: null); the disk + // assertions above are the invariant either way. + if (res.ledger) assert.strictEqual(res.ledger.id, defect.ledgerId); +}); + +ok('D2 a dedup repeat is a no-op decided before any intent; an escalation mirrors', () => { + const repo = fixture('d2'); + initStore(repo); + settled(() => artifacts.addDefect(repo, { severity: 'medium', summary: 'same finding' })); + const before = storeSnapshot(repo); + const dup = settled(() => artifacts.addDefect(repo, { severity: 'medium', summary: 'same finding' })); + assert.strictEqual(dup.deduped, true); + assert.deepStrictEqual(storeSnapshot(repo), before, 'a dedup writes no slot, no revision, no mirror'); + const esc = settled(() => artifacts.addDefect(repo, { severity: 'critical', summary: 'same finding' })); + assert.strictEqual(esc.state.severity, 'critical'); + const disk = readState(repo); + const mirror = readLedger(repo).defects.find((d) => d.id === disk.defects[0].ledgerId); + assert.strictEqual(mirror.severity, 'critical', 'the escalation reached the mirror in the same operation'); + assert.strictEqual(readLedger(repo).defects.length, 1, 'escalation replaces; it does not mint a second mirror'); +}); + +ok('D3 a legacy defect with no valid mirror is admitted on its first committed escalation', () => { + const repo = fixture('d3'); + initStore(repo); + // A pre-4b shape: defect on the record, no ledgerId, no mirror. + state.withWorkspaceMutation(repo, { action: 'seed legacy' }, (s) => { + s.defects.push({ id: 'def-legacy', at: 'old', severity: 'low', summary: 'legacy finding', status: 'open', artifact: '', attachedBy: 'none' }); + }); + const mirrorsBefore = readLedger(repo).defects.length; + settled(() => artifacts.addDefect(repo, { severity: 'high', summary: 'legacy finding' })); + const disk = readState(repo); + const legacy = disk.defects.find((d) => d.id === 'def-legacy'); + assert.strictEqual(legacy.severity, 'high'); + assert.ok(legacy.ledgerId, 'admission minted and back-linked a mirror'); + const mirror = readLedger(repo).defects.find((d) => d.id === legacy.ledgerId); + assert.strictEqual(mirror.severity, 'high'); + assert.strictEqual(mirror.summary, 'legacy finding'); + assert.strictEqual(readLedger(repo).defects.length, mirrorsBefore + 1, 'old rows untouched, one admission'); +}); + +// --------------------------------------------------------------------------- +// The crash matrix: real processes dying at the real file operations. +// --------------------------------------------------------------------------- + +const CRASH_CHILD = path.join(tmp, 'wal-crash-child.js'); +fs.writeFileSync(CRASH_CHILD, ` +'use strict'; +// argv: [node, script, repo, killPoint, mode] +// killPoint: intent | state | ledger | clear — die immediately BEFORE that +// canonical operation. mode 'recover' arms the failpoint and then triggers +// recovery (so recovery itself dies mid-flight); mode 'add' runs a defect add. +const fs = require('fs'); +const path = require('path'); +const repo = process.argv[2]; +const killPoint = process.argv[3]; +const mode = process.argv[4] || 'add'; +const orig = { rename: fs.renameSync, link: fs.linkSync, unlink: fs.unlinkSync }; +const die = () => process.exit(87); +const ends = (p, name) => String(p).replace(/\\\\/g, '/').endsWith('/' + name) || path.basename(String(p)) === name; +fs.renameSync = (a, b) => { + if (killPoint === 'state' && ends(b, 'state.json')) die(); + if (killPoint === 'ledger' && ends(b, 'ledger.json')) die(); + return orig.rename(a, b); +}; +fs.linkSync = (a, b) => { + if (killPoint === 'intent' && ends(b, 'intent.json')) die(); + return orig.link(a, b); +}; +fs.unlinkSync = (p) => { + if (killPoint === 'clear' && ends(p, 'intent.json')) die(); + return orig.unlink(p); +}; +const state = require(process.argv[5]); +const artifacts = require(process.argv[6]); +if (mode === 'recover') { + state.withWorkspaceLock(repo, 'crash-child recovery', () => {}); +} else if (mode === 'resolve') { + artifacts.transitionDefect(repo, process.argv[7], 'resolved', { evidence: 'crash resolve', note: 'resolved: crash resolve' }); +} else { + artifacts.addDefect(repo, { severity: 'high', summary: 'crash matrix finding' }); +} +process.stdout.write('SURVIVED'); +`, 'utf8'); + +function crashChild(repo, killPoint, mode, extra) { + const res = childProcess.spawnSync(process.execPath, [ + CRASH_CHILD, repo, killPoint, mode || 'add', + path.join(__dirname, '..', 'src', 'state.js'), + path.join(__dirname, '..', 'src', 'artifacts.js'), + extra || '', + ], { encoding: 'utf8', env: Object.assign({}, cleanGitEnv(), { RATCHET_LOCK_STALE_MS: '1' }), windowsHide: true }); + return res; +} + +ok('X1 death before the intent publish leaves nothing anywhere', () => { + const repo = fixture('x1'); + initStore(repo); + const before = storeSnapshot(repo); + const res = crashChild(repo, 'intent'); + assert.strictEqual(res.status, 87, res.stderr); + assert.deepStrictEqual(storeSnapshot(repo), before, 'no slot, no state, no mirror'); + const retry = settled(() => artifacts.addDefect(repo, { severity: 'high', summary: 'crash matrix finding' })); + assert.ok(retry.state.id, 'the retry applies exactly once'); + assert.strictEqual(readState(repo).defects.length, 1); +}); + +ok('X2 death before the state commit: intent discarded, retry applies once', () => { + const repo = fixture('x2'); + initStore(repo); + const res = crashChild(repo, 'state'); + assert.strictEqual(res.status, 87, res.stderr); + assert.ok(fs.existsSync(state.intentPath(repo)), 'the slot survived the death'); + assert.deepStrictEqual(state.diagnoseIntent(repo), { pending: true, verdict: 'discarded' }); + settled(() => artifacts.addDefect(repo, { severity: 'high', summary: 'crash matrix finding' })); + assert.ok(!fs.existsSync(state.intentPath(repo))); + assert.strictEqual(readState(repo).defects.length, 1, 'exactly one application'); + assert.strictEqual(readLedger(repo).defects.length, 1); +}); + +ok('X3 death between state and mirror: recovery completes the exact recorded bytes', () => { + const repo = fixture('x3'); + initStore(repo); + const res = crashChild(repo, 'ledger'); + assert.strictEqual(res.status, 87, res.stderr); + const intent = readIntent(repo); + assert.strictEqual(hashOf(state.statePath(repo)), intent.stateAfterHash, 'the decision landed'); + assert.strictEqual(hashOf(state.ledgerPath(repo)), intent.ledgerBeforeHash, 'the mirror is owed'); + triggerRecovery(repo); + assert.ok(!fs.existsSync(state.intentPath(repo))); + assert.strictEqual(hashOf(state.ledgerPath(repo)), intent.ledgerAfterHash, + 'the mirror converged to the bytes the crashed process proved before dying'); + const disk = readState(repo); + const mirror = readLedger(repo).defects.find((d) => d.id === disk.defects[0].ledgerId); + assert.strictEqual(mirror.severity, disk.defects[0].severity); +}); + +ok('X4 death before the clear: recovery clears without rewriting a byte', () => { + const repo = fixture('x4'); + initStore(repo); + const res = crashChild(repo, 'clear'); + assert.strictEqual(res.status, 87, res.stderr); + const intent = readIntent(repo); + assert.strictEqual(hashOf(state.ledgerPath(repo)), intent.ledgerAfterHash, 'mirror already landed'); + const before = storeSnapshot(repo); + triggerRecovery(repo); + delete before['intent.json']; + assert.deepStrictEqual(storeSnapshot(repo), before, 'only the slot moved'); +}); + +ok('X5 recovery killed mid-flight restarts and converges to the same bytes', () => { + const repo = fixture('x5'); + initStore(repo); + assert.strictEqual(crashChild(repo, 'ledger').status, 87, 'arm: mirror owed'); + const intent = readIntent(repo); + // Recovery dies at ITS ledger publish — the slot and the owed mirror survive. + const rec = crashChild(repo, 'ledger', 'recover'); + assert.strictEqual(rec.status, 87, rec.stderr); + assert.ok(fs.existsSync(state.intentPath(repo)), 'the slot survives a death inside recovery'); + // A second recovery death, this time at the clear: the mirror has landed. + const rec2 = crashChild(repo, 'clear', 'recover'); + assert.strictEqual(rec2.status, 87, rec2.stderr); + assert.strictEqual(hashOf(state.ledgerPath(repo)), intent.ledgerAfterHash); + triggerRecovery(repo); + assert.ok(!fs.existsSync(state.intentPath(repo))); + assert.strictEqual(hashOf(state.statePath(repo)), intent.stateAfterHash); + assert.strictEqual(hashOf(state.ledgerPath(repo)), intent.ledgerAfterHash, + 'N deaths later, the store is byte-identical to the crash-free outcome'); +}); + +// --------------------------------------------------------------------------- +// The MCP door: defect.add over the wire. +// --------------------------------------------------------------------------- + +function service(roots, write) { + return mcp.createServer({ roots, write, serverInfo: { name: 'torque-mcp-test', version: '0.0.0' } }); +} + +let requestId = 0; +function modern(conn, method, params) { + return conn.handleMessage({ + jsonrpc: '2.0', id: ++requestId, method, + params: { ...(params || {}), _meta: { + [META + 'protocolVersion']: MODERN, + [META + 'clientCapabilities']: {}, + [META + 'clientInfo']: { name: 'test-client', version: '0' }, + } }, + }); +} + +function callTool(conn, name, arguments_) { + return modern(conn, 'tools/call', { name, arguments: arguments_ }); +} + +function payload(response) { + assert.strictEqual(response.error, undefined, response.error && response.error.message); + assert.notStrictEqual(response.result.isError, true, JSON.stringify(response.result)); + assert.deepStrictEqual(response.result.structuredContent, JSON.parse(response.result.content[0].text)); + return response.result.structuredContent; +} + +function refusal(response) { + assert.strictEqual(response.error, undefined, response.error && response.error.message); + assert.strictEqual(response.result.isError, true, JSON.stringify(response.result)); + const structured = response.result.structuredContent; + assert.strictEqual(structured.message, mcp.WRITE_REFUSALS[structured.error], + 'every refusal message comes from the one allowlisted table'); + return structured; +} + +function opId() { + return crypto.randomBytes(16).toString('base64url'); +} + +function openWorkspace(conn, repo) { + return payload(callTool(conn, 'workspace.open', { path: repo })); +} + +function envelopeFor(open, extra) { + return Object.assign({ + workspaceHandle: open.workspaceHandle, + expectedStateRev: open.stateRev, + expectedStateGen: open.stateGen, + operationId: opId(), + }, extra || {}); +} + +ok('M1 defect.add commits state + mirror with derived ids; the CLI writes the same meaning', () => { + const mcpRepo = initRepo('m1-mcp'); + const cliRepo = initRepo('m1-cli'); + const conn = service([mcpRepo], true).createConnection(); + const open = openWorkspace(conn, mcpRepo); + assert.strictEqual(open.pendingIntent, false); + const result = payload(callTool(conn, 'defect.add', + envelopeFor(open, { item: { severity: 'high', summary: 'wire finding' } }))); + assert.strictEqual(result.committed, true); + assert.strictEqual(result.action, 'created'); + assert.match(result.defectId, /^def-[0-9a-f]{32}$/); + assert.match(result.ledgerId, /^ldef-[0-9a-f]{32}$/); + assert.strictEqual(result.artifact, null); + assert.ok(!fs.existsSync(state.intentPath(mcpRepo))); + const disk = readState(mcpRepo); + assert.strictEqual(disk.defects[0].ledgerId, result.ledgerId); + const mirror = readLedger(mcpRepo).defects.find((d) => d.id === result.ledgerId); + assert.strictEqual(mirror.severity, 'high'); + assert.strictEqual(disk.operations.length, 1, 'the receipt rode the state commit'); + // Same settled contract for the spawned CLI: the pending-mirror exit says + // "re-run the command", so the harness reruns it exactly once. + const cliArgs = [path.join(__dirname, '..', 'bin', 'ratchet'), 'defect', 'add', '{"severity":"high","summary":"wire finding"}']; + const cliRun = childProcess.spawnSync(process.execPath, cliArgs, + { cwd: cliRepo, encoding: 'utf8', env: cleanGitEnv(), windowsHide: true }); + if (cliRun.status !== 0) { + assert.match(String(cliRun.stderr), /mirror is pending recovery/, cliRun.stderr); + const rerun = childProcess.spawnSync(process.execPath, cliArgs, + { cwd: cliRepo, encoding: 'utf8', env: cleanGitEnv(), windowsHide: true }); + assert.strictEqual(rerun.status, 0, rerun.stderr); + } + const viaCli = readState(cliRepo).defects[0]; + const strip = (d) => ({ severity: d.severity, summary: d.summary, status: d.status, artifact: d.artifact, attachedBy: d.attachedBy }); + assert.deepStrictEqual(strip(readState(mcpRepo).defects[0]), strip(viaCli)); + const cliMirror = readLedger(cliRepo).defects.find((d) => d.id === viaCli.ledgerId); + const stripM = (m) => ({ severity: m.severity, summary: m.summary, status: m.status }); + assert.deepStrictEqual(stripM(mirror), stripM(cliMirror), 'one mirror meaning on both doors'); +}); + +ok('M2 dedup no-ops byte-purely; escalation commits and mirrors; replay answers the retry', () => { + const repo = initRepo('m2-repo'); + const conn = service([repo], true).createConnection(); + const open = openWorkspace(conn, repo); + payload(callTool(conn, 'defect.add', envelopeFor(open, { item: { severity: 'medium', summary: 'finding' } }))); + const before = storeSnapshot(repo); + const dup = payload(callTool(conn, 'defect.add', + envelopeFor({ ...open, stateRev: open.stateRev + 1 }, { item: { severity: 'medium', summary: 'finding' } }))); + assert.strictEqual(dup.committed, false); + assert.strictEqual(dup.action, 'deduped'); + assert.deepStrictEqual(storeSnapshot(repo), before, 'a dedup moves nothing'); + const escalate = envelopeFor({ ...open, stateRev: open.stateRev + 1 }, { item: { severity: 'critical', summary: 'finding' } }); + const esc = payload(callTool(conn, 'defect.add', escalate)); + assert.strictEqual(esc.action, 'escalated'); + assert.strictEqual(esc.severity, 'critical'); + const mirror = readLedger(repo).defects.find((d) => d.id === esc.ledgerId); + assert.strictEqual(mirror.severity, 'critical'); + const after = storeSnapshot(repo); + const retry = payload(callTool(conn, 'defect.add', escalate)); + assert.deepStrictEqual(retry, { ...esc, replayed: true }, 'the verbatim retry is the receipt'); + assert.deepStrictEqual(storeSnapshot(repo), after, 'a replay is a pure read'); +}); + +ok('M3 several live artifacts refuse AttachmentAmbiguous with zero bytes', () => { + const repo = initRepo('m3-repo'); + const conn = service([repo], true).createConnection(); + const open = openWorkspace(conn, repo); + payload(callTool(conn, 'artifact.add', envelopeFor(open, { item: { title: 'one' } }))); + payload(callTool(conn, 'artifact.add', envelopeFor({ ...open, stateRev: open.stateRev + 1 }, { item: { title: 'two' } }))); + const before = storeSnapshot(repo); + const refused = refusal(callTool(conn, 'defect.add', + envelopeFor({ ...open, stateRev: open.stateRev + 2 }, { item: { summary: 'homeless finding' } }))); + assert.strictEqual(refused.error, 'AttachmentAmbiguous'); + assert.deepStrictEqual(storeSnapshot(repo), before, 'the refusal moved zero bytes'); + const claimed = callTool(conn, 'defect.add', + envelopeFor({ ...open, stateRev: open.stateRev + 2 }, { item: { summary: 'x', status: 'resolved' } })); + assert.ok(claimed.error && claimed.error.code === -32602, 'a terminal birth refuses at the boundary'); +}); + +ok('M4 a post-decision mirror failure answers WriteFailed; the exact retry recovers then replays', () => { + const repo = initRepo('m4-repo'); + const conn = service([repo], true).createConnection(); + const open = openWorkspace(conn, repo); + const envelope = envelopeFor(open, { item: { severity: 'high', summary: 'interrupted finding' } }); + const origRename = fs.renameSync; + fs.renameSync = (a, b) => { + if (path.basename(String(b)) === 'ledger.json') { + const e = new Error('injected mirror failure'); + e.code = 'EIO'; + throw e; + } + return origRename(a, b); + }; + let failed; + try { + failed = refusal(callTool(conn, 'defect.add', envelope)); + } finally { + fs.renameSync = origRename; + } + assert.strictEqual(failed.error, 'WriteFailed', 'no success is emitted before mirror + clear'); + assert.ok(fs.existsSync(state.intentPath(repo)), 'the slot survives for the next writer'); + assert.strictEqual(readState(repo).operations.length, 1, 'the decision itself landed'); + const retry = payload(callTool(conn, 'defect.add', envelope)); + assert.strictEqual(retry.replayed, true, 'the retry recovers the mirror, then answers from the receipt'); + assert.ok(!fs.existsSync(state.intentPath(repo))); + const disk = readState(repo); + const mirror = readLedger(repo).defects.find((d) => d.id === disk.defects[0].ledgerId); + assert.strictEqual(mirror.severity, 'high', 'the mirror was completed by recovery'); +}); + +ok('M5 an unprovable slot refuses MirrorUnrecoverable on every write door and on open', () => { + const repo = initRepo('m5-repo'); + const conn = service([repo], true).createConnection(); + const open = openWorkspace(conn, repo); + craftedSlot(repo, 'after-state'); + const broken = readIntent(repo); + broken.ledgerAfterHash = `sha256:${'f'.repeat(64)}`; + fs.writeFileSync(state.intentPath(repo), wal.serializeRecord(broken)); + const before = storeSnapshot(repo); + const mirrored = refusal(callTool(conn, 'defect.add', envelopeFor(open, { item: { summary: 'x' } }))); + assert.strictEqual(mirrored.error, 'MirrorUnrecoverable'); + const single = refusal(callTool(conn, 'state.set', envelopeFor(open, { key: 'objective', value: 'x' }))); + assert.strictEqual(single.error, 'MirrorUnrecoverable', 'the safe-core writers refuse over the same store'); + assert.deepStrictEqual(storeSnapshot(repo), before, 'no refusal moved a byte'); + const conn2 = service([repo], true).createConnection(); + const reopened = callTool(conn2, 'workspace.open', { path: repo }); + assert.strictEqual(reopened.result.isError, true); + assert.strictEqual(reopened.result.content[0].text, mcp.WRITE_REFUSALS.MirrorUnrecoverable, + 'open speaks the same allowlisted sentence'); +}); + +ok('M6 pendingIntent is stated on every read, true under an occupied slot, reads stay pure', () => { + const repo = initRepo('m6-repo'); + const conn = service([repo], true).createConnection(); + const open = openWorkspace(conn, repo); + const stateUri = open.resources.state; + const readOnce = () => { + const res = modern(conn, 'resources/read', { uri: stateUri }); + assert.strictEqual(res.error, undefined); + return JSON.parse(res.result.contents[0].text); + }; + assert.strictEqual(readOnce().pendingIntent, false, 'a settled store says so'); + craftedSlot(repo, 'after-state'); + const before = storeSnapshot(repo); + assert.strictEqual(readOnce().pendingIntent, true, 'an occupied slot is stated, never hidden'); + assert.deepStrictEqual(storeSnapshot(repo), before, 'the read repaired nothing'); + assert.ok(!('pendingIntent' in readState(repo)), 'disk bytes never gain the flag'); +}); + +// --------------------------------------------------------------------------- +// 4b.2: the transitions ride the same slot. +// --------------------------------------------------------------------------- + +ok('T1 a CLI resolve moves state and mirror in one op; the exact repeat is a no-op; a conflicting repeat refuses', () => { + const repo = fixture('t1'); + initStore(repo); + const added = settled(() => artifacts.addDefect(repo, { severity: 'high', summary: 'transition me' })); + settled(() => artifacts.transitionDefect(repo, added.state.id, 'resolved', { evidence: 'the fix shipped', note: 'resolved: the fix shipped' })); + const disk = readState(repo); + const defect = disk.defects[0]; + assert.strictEqual(defect.status, 'resolved'); + const mirror = readLedger(repo).defects.find((d) => d.id === defect.ledgerId); + assert.strictEqual(mirror.status, 'resolved', 'the mirror followed in the same operation'); + assert.ok(!fs.existsSync(state.intentPath(repo))); + const before = storeSnapshot(repo); + const repeat = settled(() => artifacts.transitionDefect(repo, added.state.id, 'resolved', { evidence: 'the fix shipped', note: 'resolved: the fix shipped' })); + assert.strictEqual(repeat.status, 'resolved'); + assert.deepStrictEqual(storeSnapshot(repo), before, + 'the exact repeat pushes no log, no history, no revision, no intent'); + assert.throws( + () => artifacts.transitionDefect(repo, added.state.id, 'resolved', { evidence: 'a different story' }), + /different recorded proof/, + 'a conflicting repeat never silently replaces the original proof' + ); + assert.deepStrictEqual(storeSnapshot(repo), before, 'the conflicting refusal moved zero bytes'); +}); + +ok('T2 the CLI-only waive rides the WAL too; wire and internal rosters stay distinct', () => { + const repo = fixture('t2'); + initStore(repo); + const added = settled(() => artifacts.addDefect(repo, { severity: 'medium', summary: 'waive me' })); + settled(() => artifacts.transitionDefect(repo, added.state.id, 'waived', { owner: 'danny', reason: 'ships anyway', note: 'waived by danny: ships anyway' })); + const defect = readState(repo).defects[0]; + assert.strictEqual(defect.status, 'waived'); + assert.strictEqual(defect.waivedBy, 'danny'); + const mirror = readLedger(repo).defects.find((d) => d.id === defect.ledgerId); + assert.strictEqual(mirror.status, 'waived', 'the internal waiver keeps the mirror truthful'); + assert.ok(!fs.existsSync(state.intentPath(repo))); +}); + +ok('T3 a legacy defect is admitted by its first committed transition, mirror born in the new status', () => { + const repo = fixture('t3'); + initStore(repo); + state.withWorkspaceMutation(repo, { action: 'seed legacy' }, (s) => { + s.defects.push({ id: 'def-old', at: 'old', severity: 'high', summary: 'ancient finding', status: 'open', artifact: '', attachedBy: 'none' }); + }); + settled(() => artifacts.transitionDefect(repo, 'def-old', 'resolved', { evidence: 'finally fixed' })); + const defect = readState(repo).defects.find((d) => d.id === 'def-old'); + assert.ok(defect.ledgerId, 'admission minted and back-linked the mirror'); + const mirror = readLedger(repo).defects.find((d) => d.id === defect.ledgerId); + assert.strictEqual(mirror.status, 'resolved'); + assert.strictEqual(mirror.summary, 'ancient finding'); +}); + +ok('T4 death between a transition and its mirror recovers to the exact recorded bytes', () => { + const repo = fixture('t4'); + initStore(repo); + const added = settled(() => artifacts.addDefect(repo, { severity: 'high', summary: 'crash matrix finding' })); + const res = crashChild(repo, 'ledger', 'resolve', added.state.id); + assert.strictEqual(res.status, 87, res.stderr); + const intent = readIntent(repo); + assert.strictEqual(intent.tool, 'defect resolve'); + assert.strictEqual(hashOf(state.statePath(repo)), intent.stateAfterHash, 'the transition landed'); + triggerRecovery(repo); + assert.strictEqual(hashOf(state.ledgerPath(repo)), intent.ledgerAfterHash, 'the mirror converged byte-exactly'); + const defect = readState(repo).defects[0]; + assert.strictEqual(defect.status, 'resolved'); + assert.strictEqual(readLedger(repo).defects.find((d) => d.id === defect.ledgerId).status, 'resolved'); +}); + +ok('T5 the wire transitions mean the CLI meaning; replay answers the retry; supersede reason is one optional', () => { + const repo = initRepo('t5-repo'); + const conn = service([repo], true).createConnection(); + const open = openWorkspace(conn, repo); + const rev = () => readState(repo).rev; + const added = payload(callTool(conn, 'defect.add', + envelopeFor(open, { item: { severity: 'high', summary: 'wire lifecycle' } }))); + const resolveEnvelope = envelopeFor({ ...open, stateRev: rev() }, { id: added.defectId, evidence: 'proven fixed' }); + const resolved = payload(callTool(conn, 'defect.resolve', resolveEnvelope)); + assert.deepStrictEqual(resolved, { + ok: true, committed: true, stateRev: resolveEnvelope.expectedStateRev + 1, replayed: false, + defectId: added.defectId, status: 'resolved', ledgerId: added.ledgerId, + }); + assert.strictEqual(readLedger(repo).defects.find((d) => d.id === added.ledgerId).status, 'resolved'); + const retry = payload(callTool(conn, 'defect.resolve', resolveEnvelope)); + assert.deepStrictEqual(retry, { ...resolved, replayed: true }, 'the verbatim retry is the receipt'); + // The exact repeat under a FRESH operation id is the no-op, not a refusal. + const again = payload(callTool(conn, 'defect.resolve', + envelopeFor({ ...open, stateRev: rev() }, { id: added.defectId, evidence: 'proven fixed' }))); + assert.strictEqual(again.committed, false, 'an exact repeat with a new id no-ops'); + const reopened = payload(callTool(conn, 'defect.reopen', + envelopeFor({ ...open, stateRev: rev() }, { id: added.defectId, reason: 'regressed on windows' }))); + assert.strictEqual(reopened.status, 'reopened'); + const superseded = payload(callTool(conn, 'defect.supersede', + envelopeFor({ ...open, stateRev: rev() }, { id: added.defectId, by: 'art-replacement' }))); + assert.strictEqual(superseded.status, 'superseded'); + assert.strictEqual(readState(repo).defects[0].supersededBy, 'art-replacement'); + assert.strictEqual(readLedger(repo).defects.find((d) => d.id === added.ledgerId).status, 'superseded', + 'every wire transition kept the mirror truthful'); +}); + +// --------------------------------------------------------------------------- + +process.stdout.write(`\n${passed} passed, ${failures.length} failed\n`); +if (failures.length) { + process.exitCode = 1; +} diff --git a/test/mcp-write.test.js b/test/mcp-write.test.js index bc2aca6..536e3ef 100644 --- a/test/mcp-write.test.js +++ b/test/mcp-write.test.js @@ -262,10 +262,11 @@ ok('U4 derived ids keep 128 bits, are deterministic, and vary by role', () => { const ENVELOPE_KEYS = ['workspaceHandle', 'expectedStateRev', 'expectedStateGen', 'operationId']; const SESSION_VERBS = ['state.append', 'open_loop.close', 'open_loop.park', 'assumption.close', 'compile.done']; const ARTIFACT_VERBS = ['artifact.add', 'artifact.close', 'artifact.retract', 'score.aperture']; +const MIRROR_VERBS = ['defect.add', 'defect.resolve', 'defect.reopen', 'defect.supersede']; const APERTURE_DIMS = ['ambiguity', 'terrain', 'taste', 'blastRadius', 'reversibility']; const WRITE_ROSTER = [ 'workspace.open', 'workspace.scan', 'score.confidence', 'score.friction', - 'state.set', ...SESSION_VERBS, ...ARTIFACT_VERBS, + 'state.set', ...SESSION_VERBS, ...ARTIFACT_VERBS, ...MIRROR_VERBS, ]; ok('W1 a flagless server registers no write tools and cannot dispatch one', () => { @@ -915,7 +916,8 @@ ok('V1 the --write roster advertises all ten write tools with pinned contracts', ['state.append', false], ['open_loop.close', true], ['open_loop.park', true], ['assumption.close', true], ['compile.done', true], ['artifact.add', true], ['artifact.close', true], ['artifact.retract', true], - ['score.aperture', false], + ['score.aperture', false], ['defect.add', true], + ['defect.resolve', true], ['defect.reopen', true], ['defect.supersede', true], ]) { const tool = byName.get(name); assert.deepStrictEqual(tool.annotations, @@ -926,6 +928,7 @@ ok('V1 the --write roster advertises all ten write tools with pinned contracts', 'StateNotInitialized', 'StaleGeneration', 'StaleStateRev', 'OperationIdConflict', 'DeterministicIdConflict', 'UnknownRecordId', 'ArtifactClosed', 'ClosureBlocked', 'HumanAuthorityRequired', 'RetractRefused', + 'AttachmentAmbiguous', 'MirrorUnrecoverable', 'WriteFailed', ], name); } @@ -941,6 +944,12 @@ ok('V1 the --write roster advertises all ten write tools with pinned contracts', assert.deepStrictEqual(required('artifact.retract'), [...ENVELOPE_KEYS, 'id', 'reason']); assert.ok(byName.get('artifact.retract').inputSchema.properties.supersededBy); assert.deepStrictEqual(required('score.aperture'), [...ENVELOPE_KEYS, ...APERTURE_DIMS]); + assert.deepStrictEqual(required('defect.add'), [...ENVELOPE_KEYS, 'item']); + assert.deepStrictEqual(required('defect.resolve'), [...ENVELOPE_KEYS, 'id', 'evidence']); + assert.deepStrictEqual(required('defect.reopen'), [...ENVELOPE_KEYS, 'id', 'reason']); + assert.deepStrictEqual(required('defect.supersede'), [...ENVELOPE_KEYS, 'id', 'by']); + assert.ok(byName.get('defect.supersede').inputSchema.properties.reason, 'reason is optional wire surface'); + assert.ok(!byName.has('defect.waive'), 'waivers have no MCP spelling — permanently'); const success = (name) => byName.get(name).outputSchema.oneOf[0].required; const COMMON = ['ok', 'committed', 'stateRev', 'replayed']; assert.deepStrictEqual(success('state.append'), [...COMMON, 'collection', 'recordId', 'deduped']); @@ -953,6 +962,11 @@ ok('V1 the --write roster advertises all ten write tools with pinned contracts', assert.deepStrictEqual(success('artifact.retract'), [...COMMON, 'artifactId', 'status', 'supersededBy']); assert.deepStrictEqual(success('score.aperture'), [...COMMON, 'score', 'level', 'name', 'implement', 'sequence', 'mapRequired', 'dimensions', 'scope', 'recordedFog']); + assert.deepStrictEqual(success('defect.add'), [...COMMON, + 'defectId', 'severity', 'action', 'artifact', 'attachedBy', 'ledgerId']); + for (const name of ['defect.resolve', 'defect.reopen', 'defect.supersede']) { + assert.deepStrictEqual(success(name), [...COMMON, 'defectId', 'status', 'ledgerId'], name); + } // The gated constructors are not appendable — the enum itself says so. assert.deepStrictEqual(byName.get('state.append').inputSchema.properties.collection.enum, ['decisions', 'assumptions', 'openLoops', 'touchedFiles', 'history']); @@ -963,7 +977,7 @@ ok('V2 no session or artifact verb is listed or dispatchable on a flagless serve const conn = service([repo], false).createConnection(); const listed = modern(conn, 'tools/list', {}).result.tools.map((t) => t.name); const open = openWorkspace(conn, repo); - for (const tool of [...SESSION_VERBS, ...ARTIFACT_VERBS]) { + for (const tool of [...SESSION_VERBS, ...ARTIFACT_VERBS, ...MIRROR_VERBS]) { assert.ok(!listed.includes(tool), `${tool} must not be advertised`); const response = callTool(conn, 'modern', tool, envelopeFor(open, {})); assert.strictEqual(response.error && response.error.code, -32602, tool); @@ -1176,6 +1190,9 @@ ok('V9 a transition on a record that does not exist refuses UnknownRecordId with ['assumption.close', { id: 'asm-ghost', outcome: 'tested', evidence: 'e' }], ['artifact.close', { id: 'art-ghost' }], ['artifact.retract', { id: 'art-ghost', reason: 'gone' }], + ['defect.resolve', { id: 'def-ghost', evidence: 'e' }], + ['defect.reopen', { id: 'def-ghost', reason: 'r' }], + ['defect.supersede', { id: 'def-ghost', by: 'art-x' }], ]) { const structured = refusal(callTool(conn, 'modern', tool, envelopeFor(open, semantic))); assert.strictEqual(structured.error, 'UnknownRecordId', tool); @@ -1252,6 +1269,14 @@ ok('V11 malformed verb arguments refuse at the boundary with zero bytes moved', ['score.aperture', { ambiguity: '1', terrain: 0, taste: 0, blastRadius: 0, reversibility: 0 }], ['score.aperture', { terrain: 0, taste: 0, blastRadius: 0, reversibility: 0 }], ['score.aperture', { ambiguity: 0, terrain: 0, taste: 0, blastRadius: 0, reversibility: 0, extra: 1 }], + ['defect.add', { item: [] }], + ['defect.add', { item: { summary: 'x', status: 'resolved' } }], + ['defect.add', { item: { summary: 'x', status: 'waived' } }], + ['defect.resolve', { id: 'x', evidence: '' }], + ['defect.resolve', { id: '', evidence: 'e' }], + ['defect.reopen', { id: 'x', reason: ' ' }], + ['defect.supersede', { id: 'x', by: '' }], + ['defect.supersede', { id: 'x', by: 'y', reason: '' }], ]; for (const [tool, semantic] of cases) { boundaryRefusal(callTool(conn, 'modern', tool, envelopeFor(open, semantic))); @@ -1279,6 +1304,10 @@ ok('V12 every session and artifact verb answers a foreign handle with the one no ['artifact.close', { id: 'x' }], ['artifact.retract', { id: 'x', reason: 'r' }], ['score.aperture', { ambiguity: 0, terrain: 0, taste: 0, blastRadius: 0, reversibility: 0 }], + ['defect.add', { item: { summary: 'x' } }], + ['defect.resolve', { id: 'x', evidence: 'e' }], + ['defect.reopen', { id: 'x', reason: 'r' }], + ['defect.supersede', { id: 'x', by: 'y' }], ]) { messages.add(boundaryRefusal(callTool(foreign, 'modern', tool, envelopeFor(open, semantic))).message); }