diff --git a/CHANGELOG.md b/CHANGELOG.md index 72c70f3..85e9b33 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,87 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Fixed + +- **4b.3 review round (independent Codex verification, verdict "NO" on six contract + gaps — all six accepted, fixed red-first; three claim-wording findings ruled + not-defects).** All CLI-enforced, each with a falsifier (W1–W6) seen red against + `main @ b029c81` before its fix: + - **Strict slots decode strictly (W1):** `parseIntent` now uses a fatal UTF-8 decode; + a slot with invalid bytes refuses as unrecoverable instead of being lossily + normalized into an operationId nobody published — and then cleared. + - **The receipt must carry the intent's tool (W2):** `validateMirrorReceipt` compares + `receipt.tool` to `intent.tool`; a post-state slot whose tool contradicts the + receipt it names is ambiguous, never completed. + - **Mirror validity means agreement, not linkage (W3):** the spec projection carries + status, severity, AND summary. The transition mirror op now writes all three + (summary was dropped), and the exact-repeat validity check compares all three + instead of counting linked rows — a stale-but-unique mirror is now trued by one + committed repeat, then repeats no-op again. + - **Doctor observes without recovering (W4, W4b):** the WAL diagnosis runs first, and + the writability probe no longer travels the write door at all — an existing store + is probed with a scratch file instead of `initProject`, whose lock acquisition ran + recovery. Previously doctor consumed the slot and reported "no pending intent" + about the store it had just changed; round 2 showed even a reordered probe loses a + slot that lands after the diagnosis sample, so the probe is now lock-free by + construction (W4b races a slot in after the sample and proves it survives). + - **Resource reads are byte-pure, enforced (W5):** the MCP read paths (state/ledger + resources, the receipt resource, `score.confidence`) now use pure peeks that never + create, back up, or reinitialize a record; an unprovable record answers the one + allowlisted MirrorUnrecoverable sentence. Previously a resource read over a + malformed ledger wrote a `.corrupt` backup file — a write on a read path. + - **CLEAR re-checks identity (W6):** the slot delete (spec CLEAR step) re-reads the + slot and refuses if the bytes are not the bytes this pass proved — on both the + transaction's own clear and recovery's fenced clear. + - **Round 2 of the same review added three more, fixed the same way:** the peeks + decode fatally like the slot parser — a record with invalid UTF-8 answers the + allowlisted sentence instead of serving a U+FFFD-normalized projection (W7); the + stale-mirror truing commit moves ONLY the mirror — no duplicate log/history line, + no restamped proof timestamp; "commits once solely to admit it" now means exactly + that (W3 extended); and the doctor probe race above (W4b). + - **Round 3 of the same review added two more, fixed the same way:** the strict + mirrored-write reads and recovery's two record parses decode fatally like + everything else (W8, W9) — previously a lawful write over a record carrying one + invalid byte would commit a U+FFFD-normalized serialization and settle state and + ledger in permanent disagreement with no pending intent left behind; and doctor's + missing-store branch no longer reaches `initProject` at all (W4c) — a missing + store probes its nearest existing ancestor, so no first store appearing in the + sample-to-probe window can be recovered by the diagnosis tool. Doctor no longer + creates the store as a side effect; the first write does. + - **Round 4 of the same review added two more, fixed the same way:** the ordinary + loaders joined the fatal-decode rule (W10, W11) — `readJson`'s fast-path peek + answers null on undecodable bytes, and `readJsonResilient` routes invalid UTF-8 + through the same loud backup-then-reinitialize path it has always used for invalid + JSON, with the backup now preserving the exact original bytes; previously a lawful + ordinary write would silently serialize a U+FFFD-normalized record and settle the + two canonical files in disagreement with no backup and no intent. And the doctor + writability probe now proves directory creation as well as file creation (W13) — + Windows ACLs grant the two separately, the lock is a mkdir, and a file-only probe + answered "writable" where the first real write then failed. W12 additionally pins + recovery's own fatal decode against an adversarially consistent slot whose hash + certifies undecodable bytes. + - **Round 5 returned the review's YES** ("one operation, two files, one crash story — + the mirror guarantee holds"): both round-4 fixes verified, the decode sweep found + no remaining permissive parse on any canonical byte path, and the probe matrix + passed all four ACL shapes. Its one LOW — a half-failed doctor probe left its + scratch directory behind — is fixed (cleanup in a best-effort finally, W14). + - **Known limitation (parked, review round 2): the identity-checked clear is two + syscalls.** The CLEAR step re-reads and compares before deleting, exactly as the + spec words it — but compare and unlink cannot be one atomic operation through a + pathname API (no compare-and-unlink exists; same class as the 2.5 check→read gap). + A substitution landing in that window is deleted. Reaching it requires a writer + that bypasses the workspace lock — every lawful writer publishes and clears slots + under it — so this is recorded as a named limit, not repaired with machinery the + contract cannot honor. + - Ruled NOT defects, on the record: pre-commit `.tmp-` residue (documented inert by + design in 4b.1), CLI/MCP byte-equality (ids and receipts differ by ratified + design), and the MCP retry wording (the spec deliberately gives MCP the generic + `WriteFailed`; the CLI carries the "re-run" sentence). S20's loader count was + updated to the new peek — the one-snapshot contract it pins is unchanged and it + now additionally pins that reads never touch the creating loader. + + + ### Added - **Known limitation (parked, owner: Danny): a host-environment hold can stall the diff --git a/src/artifacts.js b/src/artifacts.js index 6c34694..aa7882e 100644 --- a/src/artifacts.js +++ b/src/artifacts.js @@ -239,9 +239,10 @@ function mirrorOpFor(ledger, defect, now, mintId) { const linked = String(defect.ledgerId || ''); const matches = linked ? (ledger.defects || []).filter((x) => x && x.id === linked) : []; if (matches.length === 1) { - // The mirror follows the record on both axes the transitions own; carrying - // an unchanged value is a no-op inside the same replace. - const after = { ...matches[0], severity: defect.severity, status: defect.status || 'open', updatedAt: now }; + // The mirror follows the record on every axis the spec projection names — + // status, severity, summary; carrying an unchanged value is a no-op inside + // the same replace. + const after = { ...matches[0], severity: defect.severity, summary: defect.summary, status: defect.status || 'open', updatedAt: now }; return { collection: 'defects', id: linked, mode: 'replace', after }; } const mirror = { @@ -427,8 +428,13 @@ function prepareDefectTransition(s, ledger, id, toStatus, meta, mintId) { if (from === toStatus) { const [recorded, offered] = transitionProof(d, toStatus, meta_); const linked = String(d.ledgerId || ''); + // Valid means TRUTHFUL, not merely linked: one mirror row that agrees on + // every axis the projection names. A stale-but-unique row would otherwise + // pass as valid forever, and the no-op below would refuse to repair it. + const rows = ledger ? (ledger.defects || []).filter((x) => x && x.id === linked) : []; const mirrorValid = ledger - ? linked && (ledger.defects || []).filter((x) => x && x.id === linked).length === 1 + ? Boolean(linked) && rows.length === 1 && + rows[0].status === (d.status || 'open') && rows[0].severity === d.severity && rows[0].summary === d.summary : true; if (recorded === offered && mirrorValid) { // The exact repeat, mirror already truthful: no log line, no history, no @@ -442,8 +448,12 @@ function prepareDefectTransition(s, ledger, id, toStatus, meta, mintId) { 'replace the original. Reopen it first if the recorded proof is wrong.' ); } - // Exact repeat but the mirror is missing or ambiguous: commit once solely - // to perform the D2b admission below. + // Exact repeat but the mirror is missing, ambiguous, or stale: commit once + // SOLELY to admit or true the mirror — the mirror op alone. The transition + // already happened; re-applying it would restamp proof-owned timestamps + // and append duplicate log/history lines, forging a second audit trail + // for one event (review round 2, finding 3). + return { kind: 'commit', record: d, ledgerOps: [mirrorOpFor(ledger, d, now, mintId)] }; } d.status = toStatus; diff --git a/src/cli.js b/src/cli.js index f625f63..5447a68 100644 --- a/src/cli.js +++ b/src/cli.js @@ -804,17 +804,6 @@ function cmdDoctor(cwd, asJson) { } add('every skills/*/SKILL.md has frontmatter + description', skillProblems.length === 0, skillProblems.join('; ')); - let stateDetail = ''; - let stateOk = true; - try { - state.initProject(cwd); - stateDetail = state.projectDir(cwd); - } catch (e) { - stateOk = false; - stateDetail = e.message; - } - add('state dir writable', stateOk, stateDetail); - // 4b WAL slot — read-only diagnosis, never a repair. A recoverable slot is // informational (any supported write recovers it on its way in); a slot // strict recovery cannot prove legal is the operator's, by name. @@ -836,6 +825,46 @@ function cmdDoctor(cwd, asJson) { add('WAL intent slot', false, e.message); } + // The writability probe never travels the write door — not even for a store + // that does not exist yet: initProject acquires the workspace lock, whose + // choke point RUNS recovery, and a first store created by another process + // in the sample-to-probe window would be recovered by the very tool that + // exists to report it (seen live in review round 3). An existing store is + // probed with a scratch file (named .tmp- like every inert residue); a + // missing store probes its nearest existing ancestor — the first write + // creates the store itself. + let stateDetail = ''; + let stateOk = true; + try { + const dir = state.projectDir(cwd); // still names a store conflict out loud + let probeDir = dir; + while (!fs.existsSync(probeDir)) { + const up = path.dirname(probeDir); + if (up === probeDir) break; + probeDir = up; + } + // The probe proves BOTH creations a real writer needs: a directory (the + // lock is a mkdir; a missing store starts as one) and a file inside it — + // Windows ACLs grant these separately, and a file-only probe said + // "writable" where the first real write then failed its mkdir. + const scratch = path.join(probeDir, `.doctor-probe.tmp-${process.pid.toString(36)}`); + try { + fs.mkdirSync(scratch); + fs.writeFileSync(path.join(scratch, 'probe'), 'doctor writability probe\n'); + } finally { + // Best-effort, always: a probe that failed half-way must not leave + // residue a cold-start scan could misread as an interrupted write. + try { + fs.rmSync(scratch, { recursive: true, force: true }); + } catch (_e) { /* the writability answer stands either way */ } + } + stateDetail = probeDir === dir ? dir : `${dir} (not created yet — the first write creates it)`; + } catch (e) { + stateOk = false; + stateDetail = e.message; + } + add('state dir writable', stateOk, stateDetail); + let snapOk = true; let snapDetail = ''; try { diff --git a/src/mcp/server.js b/src/mcp/server.js index c1e382a..ae85c38 100644 --- a/src/mcp/server.js +++ b/src/mcp/server.js @@ -1276,8 +1276,8 @@ function createServer(options) { // let a write that landed in between make the revision describe a // record these layers never saw — a number that certifies the wrong // bytes is worse than no number. - const snapshot = state.loadState(record.root); - const ledger = state.loadLedger(record.root); + const snapshot = state.peekState(record.root); + const ledger = state.peekLedger(record.root); // readEventsWithHealth, not readEvents: the malformed-line warning goes // to stderr, and no MCP client reads stderr. Damage that only appears // there is damage the wire silently certifies as clean. @@ -1581,12 +1581,22 @@ function createServer(options) { const available = resourceRecord(params.uri); // 4b: every resource states the WAL observation out loud. The flag is // derived at read time and injected into the projection only — the disk - // bytes never gain it. Reads stay byte-pure: the sampler never repairs. - const sampled = samplePendingIntent(available.record.root, () => { - if (available.parsed.name === 'state') return state.loadState(available.record.root); - if (available.parsed.name === 'ledger') return state.loadLedger(available.record.root); - return receipt.assemble(available.record.root); - }); + // bytes never gain it. Reads stay byte-pure: peeks, not the resilient + // loaders — a resource read must never create a record or back one up. + let sampled; + try { + sampled = samplePendingIntent(available.record.root, () => { + if (available.parsed.name === 'state') return state.peekState(available.record.root); + if (available.parsed.name === 'ledger') return state.peekLedger(available.record.root); + return receipt.assemble(available.record.root, { peek: true }); + }); + } catch (e) { + if (e && e.code === 'ERATCHETMIRROR') { + // The conservative refusal: the one allowlisted sentence, no store path. + throw rpc.rpcError(-32603, WRITE_REFUSALS.MirrorUnrecoverable); + } + throw e; + } const value = Object.assign({}, sampled.value, { pendingIntent: sampled.pendingIntent }); return withCache({ diff --git a/src/receipt.js b/src/receipt.js index 908dff1..6de4d5b 100644 --- a/src/receipt.js +++ b/src/receipt.js @@ -42,13 +42,19 @@ function authorityState(git) { return { level: 'committed-local', label: 'committed-local — HEAD is on no remote branch' }; } -function assemble(cwd = process.cwd()) { - const s = state.loadState(cwd); +function assemble(cwd = process.cwd(), opts = {}) { + // MCP resource reads peek (byte-pure, refuse on unprovable bytes); the CLI + // cold read keeps its resilient loaders — it may be the first touch ever. + const s = opts.peek ? state.peekState(cwd) : state.loadState(cwd); let ledger = {}; - try { - ledger = state.loadLedger(cwd); - } catch (_e) { - ledger = {}; + if (opts.peek) { + ledger = state.peekLedger(cwd); + } else { + try { + ledger = state.loadLedger(cwd); + } catch (_e) { + ledger = {}; + } } let events = []; diff --git a/src/state.js b/src/state.js index 265d815..9ff2ee9 100644 --- a/src/state.js +++ b/src/state.js @@ -161,7 +161,11 @@ function ensureDir(dir) { function readJson(file) { try { - return JSON.parse(fs.readFileSync(file, 'utf8')); + // Fatal decode: this is the healthy-record fast path, and a lossy read + // here hands callers a U+FFFD-normalized object that a lawful write then + // serializes. Undecodable bytes are not healthy — answer null and let the + // slow path preserve them. + return JSON.parse(new TextDecoder('utf-8', { fatal: true }).decode(fs.readFileSync(file))); } catch (_e) { return null; } @@ -272,7 +276,7 @@ function backupCorrupt(file, raw) { function readJsonResilient(file) { let raw; try { - raw = fs.readFileSync(file, 'utf8'); + raw = fs.readFileSync(file); } catch (e) { // ENOENT is the only error that means "there is no record yet". Anything // else — an ACL that denies read but permits write, a lock, EIO — means the @@ -284,9 +288,19 @@ function readJsonResilient(file) { 'refusing to reinitialize over a record that is present but unreadable. Fix access, then re-run.' ); } - if (!raw.trim()) return null; // empty file → fresh, no noisy backup + let text; try { - const parsed = JSON.parse(raw); + text = new TextDecoder('utf-8', { fatal: true }).decode(raw); + } catch (_e) { + // Invalid bytes travel the SAME preservation path as invalid JSON. The + // lossy decode used to parse them into a U+FFFD-normalized record that a + // lawful ordinary write then serialized — the two canonical files settled + // in disagreement with no backup, no intent, and no warning. + return rejectUnusable(file, raw, 'it is not valid UTF-8'); + } + if (!text.trim()) return null; // empty file → fresh, no noisy backup + try { + const parsed = JSON.parse(text); // `null`, `false`, `0`, `""` and `[]` all parse. They are not records, and the // caller reinitializes over anything falsey — so they have to travel the SAME // preservation path as malformed bytes, or a valid-but-unusable document gets @@ -760,7 +774,7 @@ function recoverPendingIntentLocked(cwd) { throw e; } }, - clearIntent: () => clearIntentFile(file), + clearIntent: (proven) => clearIntentFile(file, proven), validateMcpReceipt: validateMirrorReceipt, }); } finally { @@ -772,11 +786,28 @@ function recoverPendingIntentLocked(cwd) { // rename: a scanner holding intent.json for an instant must not fail an // operation whose work is already done. A persistent refusal throws and the // slot survives — recovery clears it later, which is exactly what it is for. -function clearIntentFile(file) { +// The spec's CLEAR step re-checks intent identity before the delete: the bytes +// removed must be the bytes this pass proved, or the delete would erase a slot +// some other story published. A mismatch is ambiguous and the slot survives. +function clearIntentFile(file, provenBytes) { const deadline = Date.now() + envMs('RATCHET_PUBLISH_TIMEOUT_MS', 10000); for (;;) { try { fenceForFile(file); + if (provenBytes) { + let current; + try { + current = fs.readFileSync(file); + } catch (e) { + if (e && e.code === 'ENOENT') return; // already cleared — done is done + throw e; + } + if (!current.equals(Buffer.from(provenBytes))) { + const err = new Error('WAL intent unrecoverable: the slot changed identity before its clear'); + err.code = 'ERATCHETMIRROR'; + throw err; + } + } fs.unlinkSync(file); return; } catch (e) { @@ -798,6 +829,7 @@ function validateMirrorReceipt(stateObj, intent) { if (hits.length !== 1) return false; const hit = hits[0]; return ( + hit.tool === intent.tool && hit.argsHash === intent.argsHash && String(hit.gen || '') === intent.stateGen && hit.rev === intent.targetStateRev && @@ -1478,7 +1510,10 @@ function readForMirror(file, what) { throw err; } try { - const parsed = JSON.parse(bytes.toString('utf8')); + // Fatal decode, same rule as the slot parser: a lossy read here would let + // a lawful write serialize a U+FFFD-normalized record and settle the two + // files in permanent disagreement with no pending intent left behind. + const parsed = JSON.parse(new TextDecoder('utf-8', { fatal: true }).decode(bytes)); if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) throw new Error('not a record'); return { bytes, parsed }; } catch (_e) { @@ -1537,8 +1572,10 @@ function runMirrored(cwd, o, action, prepare) { }; // Self-check: the slot we publish must be one our own recovery accepts — // cap included. Failing closed here costs nothing; failing open costs a - // store nobody can recover. - wal.parseIntent(Buffer.from(wal.serializeRecord(intent), 'utf8')); + // store nobody can recover. The bytes are kept: the clear below re-checks + // it deletes exactly the slot this transaction published. + const intentBytes = Buffer.from(wal.serializeRecord(intent), 'utf8'); + wal.parseIntent(intentBytes); if (!createJsonExclusive(intentPath(cwd), intent)) { const err = new Error('the intent slot is occupied — recovery should have resolved it; refusing to overwrite'); err.code = 'ERATCHETMIRROR'; @@ -1553,7 +1590,7 @@ function runMirrored(cwd, o, action, prepare) { // recovery. The operation HAS happened; only the answer must say "pending". try { writeFileAtomic(ledgerPath(cwd), ledgerAfterBytes, () => fenceForFile(ledgerPath(cwd))); - clearIntentFile(intentPath(cwd)); + clearIntentFile(intentPath(cwd), intentBytes); } catch (e) { const err = new Error( `the state change committed (rev ${s.rev}) but the mirror is pending recovery — re-run the command: ${e && e.message}` @@ -1564,6 +1601,41 @@ function runMirrored(cwd, o, action, prepare) { return { committed: true, rev: s.rev, state: s, result: prep.result }; } +// Pure reads for the MCP read paths (spec 4b: every resource read is byte-pure, +// a stable projection or a conservative refusal). Unlike the resilient loaders +// they never create, back up, or reinitialize — present-but-unprovable bytes +// refuse with the mirror code the funnel already maps to its one sentence, and +// absence after open (which initializes both records) is the same condition. +function peekCanonical(file, what) { + let raw; + try { + raw = fs.readFileSync(file); + } catch (e) { + const err = new Error(`${what} is ${e && e.code === 'ENOENT' ? 'absent' : 'unreadable'} on a read path — run ratchet doctor`); + err.code = 'ERATCHETMIRROR'; + throw err; + } + try { + // Fatal decode, same rule as the slot parser: a lossy read would serve a + // U+FFFD-normalized projection of a record nobody wrote. + const parsed = JSON.parse(new TextDecoder('utf-8', { fatal: true }).decode(raw)); + if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) throw new Error('not a record'); + return parsed; + } catch (_e) { + const err = new Error(`${what} is present but not a readable record — run ratchet doctor`); + err.code = 'ERATCHETMIRROR'; + throw err; + } +} + +function peekState(cwd) { + return peekCanonical(statePath(cwd), 'state record'); +} + +function peekLedger(cwd) { + return peekCanonical(ledgerPath(cwd), 'ledger record'); +} + function loadLedger(cwd) { const existing = readJsonResilient(ledgerPath(cwd)); if (existing) return existing; @@ -1627,6 +1699,8 @@ module.exports = { withFileLock, diagnoseIntent, loadLedger, + peekState, + peekLedger, saveLedger, makeId, proposeOnlyAgent, diff --git a/src/wal.js b/src/wal.js index 50dce58..762e867 100644 Binary files a/src/wal.js and b/src/wal.js differ diff --git a/test/mcp-server.test.js b/test/mcp-server.test.js index 873b9e0..962873f 100644 --- a/test/mcp-server.test.js +++ b/test/mcp-server.test.js @@ -663,18 +663,28 @@ ok('S20 score.confidence loads state exactly once — stateRev comes from the sc const repo = initRepo('s20-repo'); const conn = service([repo]).createConnection(); const opened = payload(openWorkspace(conn, repo)); + // 4b.3: the read path peeks (byte-pure), so the counted loader is peekState — + // and the resilient, store-creating loadState must not be touched at all. + const realPeek = state.peekState; const realLoad = state.loadState; + let peeks = 0; let loads = 0; try { + state.peekState = function counted(cwd) { + peeks++; + return realPeek.call(state, cwd); + }; state.loadState = function counted(cwd) { loads++; return realLoad.call(state, cwd); }; payload(callTool(conn, 'score.confidence', { workspaceHandle: opened.workspaceHandle })); } finally { + state.peekState = realPeek; state.loadState = realLoad; } - assert.strictEqual(loads, 1, `score.confidence read state ${loads} times; the contract is one snapshot`); + assert.strictEqual(peeks, 1, `score.confidence read state ${peeks} times; the contract is one snapshot`); + assert.strictEqual(loads, 0, 'a derived read never touches the creating loader'); }); ok('S21 journal damage is on the wire, not on stderr where no client reads', () => { diff --git a/test/mcp-wal.test.js b/test/mcp-wal.test.js index 3cf8a83..fb59aef 100644 --- a/test/mcp-wal.test.js +++ b/test/mcp-wal.test.js @@ -891,6 +891,390 @@ ok('T5 the wire transitions mean the CLI meaning; replay answers the retry; supe 'every wire transition kept the mirror truthful'); }); +// --------------------------------------------------------------------------- +// 4b.3: falsifiers from the independent review round (Codex, 2026-08-01). +// Each one was seen RED against main @ b029c81 before its fix landed. +// --------------------------------------------------------------------------- + +ok('W1 a slot whose bytes are not UTF-8 refuses instead of parsing a lossy normalization', () => { + const repo = fixture('w1'); + craftedSlot(repo); + const slotFile = state.intentPath(repo); + const bytes = Buffer.from(bytesOf(slotFile)); + const at = bytes.indexOf(Buffer.from('wal-crafted-operation', 'utf8')); + assert.ok(at > 0, 'the crafted operationId is in the slot'); + bytes[at + 4] = 0xff; // one invalid byte inside a JSON string + fs.writeFileSync(slotFile, bytes); + const before = storeSnapshot(repo); + assert.throws(() => triggerRecovery(repo), (e) => Boolean(e) && e.code === 'ERATCHETMIRROR', + 'invalid bytes are ambiguous — a U+FFFD substitute is a different operationId, not this one'); + assert.deepStrictEqual(storeSnapshot(repo), before, 'the refusal preserved every byte'); +}); + +ok('W2 recovery refuses a post-state slot whose tool contradicts the receipt it names', () => { + const repo = initRepo('w2'); + const conn = service([repo], true).createConnection(); + const open = openWorkspace(conn, repo); + const origRename = fs.renameSync; + fs.renameSync = (a, b) => { + if (path.basename(String(b)) === 'ledger.json') { + const e = new Error('injected mirror failure'); + e.code = 'EIO'; + throw e; + } + return origRename(a, b); + }; + let failed; + try { + failed = refusal(callTool(conn, 'defect.add', envelopeFor(open, { item: { severity: 'high', summary: 'tool binding' } }))); + } finally { + fs.renameSync = origRename; + } + assert.strictEqual(failed.error, 'WriteFailed'); + const slot = readIntent(repo); + assert.strictEqual(slot.tool, 'defect.add'); + slot.tool = 'defect.resolve'; // a known tool — just not the one this receipt earned + fs.writeFileSync(state.intentPath(repo), wal.serializeRecord(slot)); + const before = storeSnapshot(repo); + assert.throws(() => triggerRecovery(repo), (e) => Boolean(e) && e.code === 'ERATCHETMIRROR', + 'an after-image receipt must carry the tool the intent names'); + assert.deepStrictEqual(storeSnapshot(repo), before, 'nothing moved under the contradiction'); +}); + +ok('W3 an exact repeat over a stale mirror commits once and trues status, severity, and summary', () => { + const repo = fixture('w3'); + initStore(repo); + const added = settled(() => artifacts.addDefect(repo, { severity: 'high', summary: 'truth drifts' })); + settled(() => artifacts.transitionDefect(repo, added.state.id, 'resolved', { evidence: 'fixed', note: 'resolved: fixed' })); + const ledgerId = readState(repo).defects[0].ledgerId; + const settledState = readState(repo); + const audit = { + history: settledState.history.length, + log: settledState.defects[0].log.length, + resolvedAt: settledState.defects[0].resolvedAt, + }; + const ledger = readLedger(repo); + const row = ledger.defects.find((d) => d.id === ledgerId); + row.status = 'open'; + row.severity = 'low'; + row.summary = 'STALE SUMMARY'; + fs.writeFileSync(state.ledgerPath(repo), wal.serializeRecord(ledger)); + const repeat = settled(() => artifacts.transitionDefect(repo, added.state.id, 'resolved', { evidence: 'fixed', note: 'resolved: fixed' })); + assert.strictEqual(repeat.status, 'resolved'); + const truedState = readState(repo); + assert.deepStrictEqual( + { history: truedState.history.length, log: truedState.defects[0].log.length, resolvedAt: truedState.defects[0].resolvedAt }, + audit, + 'the truing commit moved ONLY the mirror — no duplicate audit line, no restamped proof'); + const trued = readLedger(repo).defects.find((d) => d.id === ledgerId); + assert.deepStrictEqual( + { status: trued.status, severity: trued.severity, summary: trued.summary }, + { status: 'resolved', severity: 'high', summary: 'truth drifts' }, + 'the one committed repeat trues every axis the spec projection names'); + const settledBytes = storeSnapshot(repo); + settled(() => artifacts.transitionDefect(repo, added.state.id, 'resolved', { evidence: 'fixed', note: 'resolved: fixed' })); + assert.deepStrictEqual(storeSnapshot(repo), settledBytes, 'once truthful, the repeat is a no-op again'); +}); + +ok('W4 doctor observes a pending slot without recovering it', () => { + const repo = fixture('w4'); + craftedSlot(repo, 'after-state'); // recoverable: the mirror is owed + const before = storeSnapshot(repo); + const run = childProcess.spawnSync(process.execPath, [path.join(__dirname, '..', 'bin', 'ratchet'), 'doctor'], + { cwd: repo, encoding: 'utf8', env: cleanGitEnv(), windowsHide: true }); + assert.match(String(run.stdout), /pending/i, 'doctor names the slot it sees'); + assert.deepStrictEqual(storeSnapshot(repo), before, 'diagnosis moved zero bytes — doctor never recovers'); +}); + +ok('W4b a slot landing after the diagnosis sample still survives doctor', () => { + const repo = initRepo('w4b'); + const cli = require('../src/cli'); + const realDiagnose = state.diagnoseIntent; + const realWrite = process.stdout.write; + const realExit = process.exitCode; + const realCwd = process.cwd(); + let planted = null; + try { + state.diagnoseIntent = function raced(cwd) { + const res = realDiagnose.call(state, cwd); + planted = craftedSlot(repo, 'after-state'); // the slot lands right after the sample + return res; + }; + process.chdir(repo); + process.stdout.write = () => true; // doctor's report is not under test; its writes are + cli.run(['node', 'ratchet', 'doctor', '--json']); + } finally { + process.stdout.write = realWrite; + process.chdir(realCwd); + state.diagnoseIntent = realDiagnose; + process.exitCode = realExit; + } + assert.ok(fs.existsSync(state.intentPath(repo)), 'the late slot is not consumed by the probe'); + assert.strictEqual(hashOf(state.ledgerPath(repo)), planted.intent.ledgerBeforeHash, + 'the owed mirror is still owed — doctor recovered nothing'); +}); + +ok('W4c doctor never travels the write door — a missing store is probed, not initialized', () => { + const repo = initRepo('w4c'); // no initStore: the store directory does not exist + const cli = require('../src/cli'); + const realInit = state.initProject; + const realWrite = process.stdout.write; + const realExit = process.exitCode; + const realCwd = process.cwd(); + let entered = 0; + let captured = ''; + try { + state.initProject = function counted(...a) { + entered++; + return realInit.apply(state, a); + }; + process.chdir(repo); + process.stdout.write = (s) => { + captured += String(s); + return true; + }; + cli.run(['node', 'ratchet', 'doctor', '--json']); + } finally { + process.stdout.write = realWrite; + process.chdir(realCwd); + state.initProject = realInit; + process.exitCode = realExit; + } + const writable = JSON.parse(captured).checks.find((c) => c.name === 'state dir writable'); + assert.strictEqual(entered, 0, + 'the probe must never enter initProject — its lock would recover whatever store just appeared'); + assert.strictEqual(writable.ok, true, 'the writability answer is still a real answer'); + assert.ok(!fs.existsSync(state.projectDir(repo)), 'diagnosis created no store'); +}); + +ok('W8 a mirrored write over a state record with invalid UTF-8 refuses and moves nothing', () => { + const repo = fixture('w8'); + initStore(repo); + settled(() => artifacts.addDefect(repo, { severity: 'high', summary: 'utf8 sentinel state' })); + const stateFile = state.statePath(repo); + const bytes = Buffer.from(bytesOf(stateFile)); + const at = bytes.indexOf(Buffer.from('utf8 sentinel state', 'utf8')); + assert.ok(at > 0); + bytes[at + 2] = 0xff; // one invalid byte inside the recorded summary + fs.writeFileSync(stateFile, bytes); + const before = storeSnapshot(repo); + assert.throws( + () => artifacts.addDefect(repo, { severity: 'low', summary: 'a different lawful finding' }), + (e) => Boolean(e) && e.code === 'ERATCHETMIRROR', + 'a strict read never normalizes canonical bytes into a record nobody wrote'); + assert.deepStrictEqual(storeSnapshot(repo), before, 'the refusal moved zero bytes'); +}); + +ok('W9 a mirrored write over a ledger record with invalid UTF-8 refuses and moves nothing', () => { + const repo = fixture('w9'); + initStore(repo); + settled(() => artifacts.addDefect(repo, { severity: 'high', summary: 'utf8 sentinel ledger' })); + const ledgerFile = state.ledgerPath(repo); + const bytes = Buffer.from(bytesOf(ledgerFile)); + const at = bytes.indexOf(Buffer.from('utf8 sentinel ledger', 'utf8')); + assert.ok(at > 0); + bytes[at + 2] = 0xff; + fs.writeFileSync(ledgerFile, bytes); + const before = storeSnapshot(repo); + assert.throws( + () => artifacts.addDefect(repo, { severity: 'low', summary: 'a different lawful finding' }), + (e) => Boolean(e) && e.code === 'ERATCHETMIRROR', + 'the mirror side is held to the same strict decode as the state side'); + assert.deepStrictEqual(storeSnapshot(repo), before, 'the refusal moved zero bytes'); +}); + +ok('W7 a peeked record with invalid UTF-8 refuses instead of serving a normalized projection', () => { + const repo = initRepo('w7'); + const conn = service([repo], true).createConnection(); + const open = openWorkspace(conn, repo); + const stateFile = state.statePath(repo); + const bytes = Buffer.from(bytesOf(stateFile)); + const at = bytes.indexOf(Buffer.from('objective', 'utf8')); + assert.ok(at > 0, 'the state record carries an objective field'); + bytes[at + 3] = 0xff; // one invalid byte inside a JSON string + fs.writeFileSync(stateFile, bytes); + const res = modern(conn, 'resources/read', { uri: open.resources.state }); + assert.ok(res.error, 'an unprovable record is a refusal, not a lossy U+FFFD projection'); + assert.strictEqual(res.error.message, mcp.WRITE_REFUSALS.MirrorUnrecoverable, + 'the refusal speaks the allowlisted sentence, never a store path'); + assert.ok(bytes.equals(bytesOf(stateFile)), 'the read changed nothing'); +}); + +ok('W5 a resource read over an unreadable ledger refuses conservatively and writes nothing', () => { + const repo = initRepo('w5'); + const conn = service([repo], true).createConnection(); + const open = openWorkspace(conn, repo); + craftedSlot(repo); // occupy the slot so the store is mid-story + fs.writeFileSync(state.ledgerPath(repo), '{ this is not a record'); + const files = () => fs.readdirSync(state.projectDir(repo)).filter((n) => n !== '.lock' && !n.includes('.tmp-')).sort(); + const filesBefore = files(); + const ledgerBytes = bytesOf(state.ledgerPath(repo)); + const res = modern(conn, 'resources/read', { uri: open.resources.ledger }); + assert.ok(res.error, 'an unprovable ledger is a refusal, not a fresh projection'); + assert.strictEqual(res.error.message, mcp.WRITE_REFUSALS.MirrorUnrecoverable, + 'the refusal speaks the allowlisted sentence, never a store path'); + assert.deepStrictEqual(files(), filesBefore, 'the read created no backup, no fresh record, no residue'); + assert.ok(ledgerBytes.equals(bytesOf(state.ledgerPath(repo))), 'the unreadable bytes are untouched'); +}); + +ok('W6 clear verifies the slot it deletes is the slot it proved', () => { + const repo = fixture('w6'); + craftedSlot(repo); // before/before: recovery would discard, then clear + const slotFile = state.intentPath(repo); + const substitute = Object.assign(readIntent(repo), { operationId: 'wal-substituted-operation' }); + const origRead = fs.readFileSync; + let swapped = false; + fs.readFileSync = function (file, ...rest) { + const out = origRead.call(fs, file, ...rest); + if (!swapped && String(file) === slotFile) { + swapped = true; // swap AFTER recovery has read and proven the original + fs.writeFileSync(slotFile, wal.serializeRecord(substitute)); + } + return out; + }; + try { + assert.throws(() => triggerRecovery(repo), (e) => Boolean(e) && e.code === 'ERATCHETMIRROR', + 'deleting bytes that no longer match the proven slot is ambiguous, not a clear'); + } finally { + fs.readFileSync = origRead; + } + assert.strictEqual(readIntent(repo).operationId, 'wal-substituted-operation', + 'the substituted slot survived the refused clear'); +}); + +ok('W10 an ordinary state writer never serializes a normalized record — bytes back up, loudly', () => { + const repo = fixture('w10'); + initStore(repo); + settled(() => artifacts.addDefect(repo, { severity: 'high', summary: 'ordinary state sentinel' })); + const stateFile = state.statePath(repo); + const damaged = Buffer.from(bytesOf(stateFile)); + const at = damaged.indexOf(Buffer.from('ordinary state sentinel', 'utf8')); + assert.ok(at > 0); + damaged[at + 2] = 0xff; + fs.writeFileSync(stateFile, damaged); + settled(() => state.withWorkspaceMutation(repo, { action: 'w10 ordinary write' }, (s) => { + s.objective = 'a lawful unrelated write'; + })); + assert.ok(!bytesOf(stateFile).toString('utf8').includes('�'), + 'no U+FFFD ever reaches a canonical file — normalization is not recovery'); + const dir = path.dirname(stateFile); + const backups = fs.readdirSync(dir).filter((n) => n.startsWith('state.json.corrupt.')); + assert.ok(backups.length >= 1, 'the undecodable bytes were preserved before anything replaced them'); + for (const b of backups) { + assert.ok(damaged.equals(bytesOf(path.join(dir, b))), 'the backup is the exact original bytes'); + } +}); + +ok('W11 an ordinary ledger writer never serializes a normalized record — bytes back up, loudly', () => { + const repo = fixture('w11'); + initStore(repo); + settled(() => artifacts.addDefect(repo, { severity: 'high', summary: 'ordinary ledger sentinel' })); + const ledgerFile = state.ledgerPath(repo); + const damaged = Buffer.from(bytesOf(ledgerFile)); + const at = damaged.indexOf(Buffer.from('ordinary ledger sentinel', 'utf8')); + assert.ok(at > 0); + damaged[at + 2] = 0xff; + fs.writeFileSync(ledgerFile, damaged); + const loaded = state.loadLedger(repo); + settled(() => state.saveLedger(repo, loaded)); + assert.ok(!bytesOf(ledgerFile).toString('utf8').includes('�'), 'no U+FFFD ever reaches a canonical file'); + const dir = path.dirname(ledgerFile); + const backups = fs.readdirSync(dir).filter((n) => n.startsWith('ledger.json.corrupt.')); + assert.ok(backups.length >= 1, 'the undecodable bytes were preserved'); + for (const b of backups) { + assert.ok(damaged.equals(bytesOf(path.join(dir, b))), 'the backup is the exact original bytes'); + } +}); + +ok('W12 recovery refuses a state after-image whose bytes are not UTF-8, hash match or not', () => { + const repo = fixture('w12'); + craftedSlot(repo, 'after-state'); + const stateFile = state.statePath(repo); + const damaged = Buffer.from(bytesOf(stateFile)); + const at = damaged.indexOf(Buffer.from('crafted', 'utf8')); + assert.ok(at > 0); + damaged[at + 2] = 0xff; + fs.writeFileSync(stateFile, damaged); + const slot = readIntent(repo); + slot.stateAfterHash = wal.hashBytes(damaged); // an adversarially consistent slot + fs.writeFileSync(state.intentPath(repo), wal.serializeRecord(slot)); + const before = storeSnapshot(repo); + assert.throws(() => triggerRecovery(repo), (e) => Boolean(e) && e.code === 'ERATCHETMIRROR', + 'a hash can certify damaged bytes; the parse must still refuse them'); + assert.deepStrictEqual(storeSnapshot(repo), before, 'the refusal preserved every byte'); +}); + +ok('W13 the doctor probe proves directory creation, not just file creation', () => { + const repo = initRepo('w13'); + const cli = require('../src/cli'); + const realMkdir = fs.mkdirSync; + const realWrite = process.stdout.write; + const realExit = process.exitCode; + const realCwd = process.cwd(); + let captured = ''; + try { + fs.mkdirSync = function denied(target, ...rest) { + if (String(target).includes('.doctor-probe')) { + const e = new Error(`EPERM: operation not permitted, mkdir '${target}'`); + e.code = 'EPERM'; + throw e; // the ACL shape Windows grants: files allowed, directories denied + } + return realMkdir.call(fs, target, ...rest); + }; + process.chdir(repo); + process.stdout.write = (s) => { + captured += String(s); + return true; + }; + cli.run(['node', 'ratchet', 'doctor', '--json']); + } finally { + fs.mkdirSync = realMkdir; + process.stdout.write = realWrite; + process.chdir(realCwd); + process.exitCode = realExit; + } + const writable = JSON.parse(captured).checks.find((c) => c.name === 'state dir writable'); + assert.strictEqual(writable.ok, false, + 'a place where directories cannot be created is not writable for a store'); +}); + +ok('W14 a half-failed doctor probe cleans up after itself', () => { + const repo = initRepo('w14'); + initStore(repo); + const cli = require('../src/cli'); + const realWriteFile = fs.writeFileSync; + const realWrite = process.stdout.write; + const realExit = process.exitCode; + const realCwd = process.cwd(); + let captured = ''; + try { + fs.writeFileSync = function denied(target, ...rest) { + if (String(target).includes('.doctor-probe')) { + const e = new Error(`EACCES: permission denied, open '${target}'`); + e.code = 'EACCES'; + throw e; // the mkdir succeeded; the file inside is what fails + } + return realWriteFile.call(fs, target, ...rest); + }; + process.chdir(repo); + process.stdout.write = (s) => { + captured += String(s); + return true; + }; + cli.run(['node', 'ratchet', 'doctor', '--json']); + } finally { + fs.writeFileSync = realWriteFile; + process.stdout.write = realWrite; + process.chdir(realCwd); + process.exitCode = realExit; + } + const writable = JSON.parse(captured).checks.find((c) => c.name === 'state dir writable'); + assert.strictEqual(writable.ok, false, 'the half-failed probe still answers honestly'); + const residue = fs.readdirSync(state.projectDir(repo)).filter((n) => n.includes('.doctor-probe')); + assert.deepStrictEqual(residue, [], + 'a failed diagnostic leaves nothing a cold-start scan could misread as an interrupted write'); +}); + // --------------------------------------------------------------------------- process.stdout.write(`\n${passed} passed, ${failures.length} failed\n`);