Record the unknown-exception surface inventory and guard new raw flows - #2470
Conversation
Inventory every surface where an unknown (non-domain) exception can reach a user, an LLM transcript, or a durable row: standard HTTP, multi-status/batch receipts, persisted agent/command/capture/webhook state, every MCP tool and resource, the standalone CLI, SignalR, and provider health. Each row cites file:line at a1ed795, the sanitizing mechanism, a classification (safe / deliberate-domain / open residual / owned-elsewhere), and the regression test that pins it. Seven open residuals are recorded, including the legacy persisted FailureReason replay policy note from the #2432 / #2436 release comments. Part of #2351.
Add scripts/check-unknown-exception-boundary.mjs, a narrow allowlist-based guard over the two regions where an unknown-exception regression would be silent: - MCP payloads (backend/src/Taskdeck.Api/Mcp/*.cs) may not return or throw a raw .ErrorMessage without a sanitizer on the same statement. - Persisted failure state (AgentRuntime.cs, OpsCliService.cs) may not carry exception text out of a catch (Exception ex) block. Log statements and known-domain catches are never inspected, so the guard does not flag the deliberate domain-message paths the trust model relies on. One allowlist entry covers CaptureResources' replay of an already-sanitized stored value; a test enforces that every entry states a path, pattern, issue and reason. The test file rejects a synthetic snippet per rule, accepts the sanitized form of each, refuses to flag a log statement and a catch (DomainException) block, and runs the guard over the real tree expecting zero findings. Part of #2351.
…ance Add one step to the docs-governance job next to the existing governance checks; ci/policy.v1.json already routes scripts/check-*.mjs to that lane, so no policy change is needed. Record the guard commands in the policy doc's Verification section. Part of #2351.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Review round on #2470 found three precision gaps, all MEDIUM: - Rule 1 judged sanitization statement-wide, so a raw member passed when a sibling member in the same anonymous object was sanitized. It now walks the enclosing call chain of each .ErrorMessage occurrence and accepts it only when a sanitizing call actually wraps it. The reviewed ProposalTools guarded ternary stays accepted as a named shape. - Allowlist entries were matched against the whole statement, so a new raw member added inside the allowlisted 14-line Serialize statement in CaptureResources would have been silently accepted. Entries are now keyed to a path plus the exact source line. - Rule 2 scanned only the two Application files, so a new MCP tool writing catch (Exception ex) { return Error(ex.Message); } fired nothing, contrary to the header. It now covers backend/src/Taskdeck.Api/Mcp/*.cs with the same logging exemption. The existing bare catches there are exempt on their merits (they log only and report the exception type name), not by allowlist. Header wording aligned with what the rules do. Literal masking now preserves interpolation holes, so a raw value inside $"...{x}" is seen rather than blanked. Part of #2351.
Review round 1 — dispositionsNew head: 1. MEDIUM — statement-wide 2. MEDIUM — statement-wide allowlist matching. FIXED. 3. MEDIUM — header overclaimed; Also folded in while touching the masking code: literal masking now preserves interpolation holes, so LOW, declined as directed — no change. Verification at
|
|
Fresh-context independent review, round 1 at 3ef3f3f, with a scoped verification pass on the fix diff 3ef3f3f..f777c63. Verdict after the fix round: no CRITICAL or HIGH finding. Merge-blocking: none. Round count: one review round, one fix round. Round 1 findings and disposition (all three MEDIUM, all fixed):
Fix-diff verification (by trace): the four adversarial shapes behave as intended (unsanitized wrapper flagged, concatenation flagged, interpolated sanitizer call accepted, half-guarded ternary flagged); interpolation-hole preservation does not break statement splitting; the widened rule 2 flags the reviewer's snippet and exempts the telemetry catches on their merits. Worker-run proof at f777c63: guard tests 24/24, real tree 0 findings with no new allowlist entry, docs governance and golden principles pass. LOW, tracked in a follow-up issue rather than fixed here: the guarded-ternary branch is dead against the real tree and would also excuse null-conditional or named-argument shapes if it fired; rule 2's sanitized check is still statement-level and now includes LogSanitizer, so a concatenation with a sanitized prefix inside a catch is excused; a value laundered through an intermediate variable is invisible to rule 1 (stated in the doc). Remaining gate: exact-head hosted Docs Governance green (the workflow change is R4-class, hosted-only) plus the rest of ci-required, and the three-minute aging floor, then a merge commit. Part of #2351; the umbrella stays open. |
|
Current-base refresh at exact head 7918617: merged origin/main once (PRs #2466, #2471, #2474, #2416 landed). The only conflict was the Verification section of docs/security/SECURITY_LOGGING_REDACTION.md, where #2471's proof block and this PR's guard block were appended at the same point; both are kept with balanced fences. No logic changed. Post-refresh proof: guard tests 24/24, real-tree guard 0 findings on the new main (which now includes the generalized webhook worker and the CLI boundary), docs governance and diff check clean. Remaining gate: exact-head hosted CI and the aging floor. |
Summary
Two acceptance bullets of #2351: the reviewed surface inventory, and a focused guard that blocks
new raw unknown-exception flows without flagging log statements or known-domain catches.
docs/security/UNKNOWN_EXCEPTION_SURFACE_INVENTORY.mdrecords one row per surface — standard HTTP(middleware +
Resultmapper), multi-status/batch receipts, persisted agent/command/capture/webhookstate, housekeeping workers, every file under
backend/src/Taskdeck.Api/Mcp, the standalone CLI,SignalR, provider health, and the circuit-breaker snapshot. Each row cites
file:lineata1ed795a7, names the sanitizing mechanism, carries a classification(safe / deliberate-domain / open residual / owned-elsewhere), and names the regression test that
pins it. Every row was derived by reading the code, not from release notes.
scripts/check-unknown-exception-boundary.mjsis the guard, wired into the existingdocs-governanceCI job.Open residuals recorded by the inventory
These are recorded, not fixed, by this PR. None is silenced by the guard: each sits outside the two
regions the guard inspects, so no allowlist entry suppresses it.
OutboundWebhookDeliveryWorker.cs:245-248persistsRedact($"Webhook delivery threw {ex.GetType().Name}: {ex.Message}")intoOutboundWebhookDelivery.LastErrorMessage.Redactispattern-based, so any exception text that matches no rule in
SensitiveDataRedactor's replacementrules is stored verbatim (capped at 1000 chars by
OutboundWebhookDelivery.cs:146-154). Every otherreviewed persisted surface generalizes instead.
ProposalTools.Error(Result)diverges fromSanitizeLlmFailureMessage.ProposalTools.cs:187-196handles theUnexpectedErrorcode correctly but returnsresult.ErrorMessagefor every other code without callingRedact, unlikeReadTools.cs:154and
WriteTools.cs:557, which route throughSanitizeLlmFailureMessage.FailureReasonvalues are replayed unsanitized. Recorded as the standingpolicy note from the
#2432/#2436release comments: those PRs sanitized the write path fornew failures, but rows written before them, and the read projections at
ProposalResources.cs:135,ProposalTools.cs:71,Services/AutomationProposalService.cs:1591,AgentRuntime.cs:278,322andServices/AgentRunService.cs:107,129, still surface storedFailureReasontext verbatim. Historicrows are not backfilled.
SignalRRegistration.cs:20calls
AddSignalR()with no options delegate and nothing inbackend/srcsetsEnableDetailedErrors. The behaviour is correct today only because the framework default isfalse; no configuration key or test pins it, so a future options delegate could flip it silently.CircuitBreakerStateTracker.LastFailureReasonis populated fromoutcome.Exception?.Message(
LlmProviderRegistration.cs:425,459,AuthenticationRegistration.cs:289). In-memory only, but itis read back into operator-visible state snapshots.
SensitiveDataRedactor.GenericUnexpectedFailureMessage("Unexpected processing error. Check server logs with the correlation ID.") and the three private
GenericUnexpectedErrorMessageconstants ("An unexpected error occurred.") inUnhandledExceptionMiddleware.cs:11,AutomationExecutorService.cs:13andBatchProposalExecutionService.cs:11present two different strings for the same condition.Taskdeck.Cli/Program.cshas no top-leveltry/catch on this base; fix in flight as Sanitize standalone CLI unexpected failures #2466.
Guard rules
Allowlist-based and deliberately narrow. Statements, not lines, are the unit of judgement, so a
multi-line
return JsonSerializer.Serialize(new { ... })is evaluated as one expression.mcp-error-message— inbackend/src/Taskdeck.Api/Mcp/*.cs, a statement in areturn/throw/Error(...)/JsonSerializer.Serialize(...)position that references.ErrorMessagemust also carry a sanitizer token (
SanitizeLlmFailureMessage,GenericUnexpectedFailureMessage,PublicFailureMessage,SummarizeException, or anySensitiveDataRedactor.call).persisted-unknown-failure— inPERSISTED_STATE_FILES(AgentRuntime.cs,OpsCliService.cs), a statement inside acatch (Exception <var>)block may not reference<var>.Message,<var>.StackTraceor<var>.ToString()unless it is a logging call or carries asanitizer token.
catch (DomainException ex)is never matched — the catch filter is what makesthose messages curated — and
_logger/Log*calls are never flagged.CaptureResources.cserrorMessage = c.ErrorMessage(Sanitize MCP capture and board resource failures #2443), safe because the write side sanitizes(
LlmQueueToProposalWorker.cs:698,TranscriptTriageWorker.cs:347). A test enforces that everyallowlist entry states a path, a pattern, an issue and a reason. A new persisted-state surface
joins the guard by being added to
PERSISTED_STATE_FILESin the PR that introduces it.Part of #2351. This PR intentionally does not close the umbrella issue.
Verification
All run in the worktree at head
3ef3f3fbf.node --test scripts/check-unknown-exception-boundary.test.mjs— 17 tests, 17 pass, 0 fail.Covers a rejected synthetic snippet per rule (raw
Error(result.ErrorMessage), rawnew { error = ... }, raw interpolation into a thrown MCP exception,ex.Messagepersisted,ex.ToString()assigned, exception text interpolated into a returned failure), the sanitizedcounterpart of each, a log statement it must not flag, a
catch (DomainException ex)block it mustnot flag, a nested save-failure logging catch, the allowlist shape, and a real-tree scan.
node scripts/check-unknown-exception-boundary.mjs— passed, 0 findings on the real tree.node scripts/check-docs-governance.mjs— passed.node scripts/check-golden-principles.mjs— passed.node --test scripts/ci/smart-ci/*.test.mjs— 91 tests, 91 pass, 0 fail (workflow touched)..github/workflows/reusable-docs-governance.yml— parses, 10 steps, the newValidate unknown-exception boundary invariantsstep present.git diff --check— clean.ci/policy.v1.jsonalready routesscripts/check-*.mjsandscripts/check-*.test.mjsto thedocs-governancelane (governance-scripts, R2), so no policy change was needed. The workflowchange adds a
run:step only, so theaction-pinscontract test is unaffected.Documentation
docs/security/UNKNOWN_EXCEPTION_SURFACE_INVENTORY.md.docs/security/SECURITY_LOGGING_REDACTION.md— Verification section gains the two guard commandsand a pointer to the inventory (append-only; no existing text changed).
docs/STATUS.md, masterplan or ADR change: this PR adds documentation and a CI check, andchanges no shipped runtime behaviour.
Not verified
.claude/rules/ci-control.mdthe CI-controlchange is R4-class and hosted-only — the local PyYAML parse and the smart-ci planner tests are
additive, not the proving check. The
docs-governancejob result on this head is the real proof.dotnet testrun. No C# was changed; the inventory's cited line numbers andtest names were read from source and from
backend/tests, not re-executed.message laundered through an intermediate variable across statements, or through a helper in a file
outside the two guarded regions.
by this PR.