Repository navigation
Redact secrets from tool output before it reaches the model - #82
Merged
Merged
Conversation
Add Provenance and ToolOutcome::External. webfetch, websearch, MCP tool calls, the browser and MCP resources (tool and @server:uri mentions) declare provenance; the registry projection truncates, exposes hidden characters, neutralizes frame tags and wraps the result in <untrusted_content>, with the truncation notice after the closing tag. The webfetch summarizer gets the page in the same frame. Overflow files of framed results record provenance in a sidecar, and read/grep of the managed output directory is re-framed, failing closed to source=overflow.
A retryable failure (transport, idle timeout, in-stream overload, rate limit) after partial text, reasoning or tool calls now discards the attempt and re-sends the same request within the existing retry budget, instead of settling the Turn as interrupted. Replay ends at protocol completion; tool calls only dispatch after the stream completes. - cyber-llm: open_with_retry returns a replaying stream that yields LlmEvent::Discarded before the new attempt; retries before and after output share one budget. collect drops discarded output, keeps usage. - Drain records reasoning items and tool calls with the text when the stream ends, so nothing of a discarded attempt reaches history. Usage a discarded attempt reported is recorded as a failed turn usage.recorded.1 unless a reservation already billed it. - session.retry carries message_id and discarded; the TUI and web dashboard clear the partial answer, exec stream-json prints a retry line. - OpenSpec change retry-interrupted-streams (Retry policy, eager execution durability, exec stream-json schema), docs and ROADMAP.
Add the versioned injection rule table and detector in cyber-core (NFKC, lower case, hidden characters removed, whitespace collapsed; seven kinds; ten recorded hits with redacted excerpts, all counted). It lives in cyber-core so the runtime can scan admitted input. The registry scans each framed result's model-facing text and adds a signals attribute; a webfetch summary carries the page's signals too. Cross-session message text, teammate idle status lines and any channel-sourced admission are scanned at admission. With permissions.untrusted_content.scan "all", read/grep/bash/powershell output is scanned without being rewritten. session.tool.settled.1 and session.prompt.admitted.1 gain an optional untrusted record (also projected on CallState and InboxRow); old events read as no signals. OpenAPI and the TypeScript SDK are regenerated. A std-only cargo bench checks the 5 ms / 50 KiB target in CI with a 2x alarm.
…ashboard Shared notice wording and per-Mode consequence live in cyber-core (untrusted::notice). The TUI draws the warning line under a tool result or promoted message, with excerpts while tool details are expanded. cyber exec text mode writes a stderr warning (suppressed by --quiet); stream-json tool_result carries the untrusted record under schema version 1. The dashboard renders the line and excerpts on tool cards and messages with textContent. Promoted messages carry their admission's record on the message listing; OpenAPI, the SDK and web/dist are regenerated. Clients state the consequence for the default taint/bypass settings until the config loader exposes them (task 5.1).
… actions Fold the run's taint into Session state from settled calls and promoted inbox rows: records since the last source "user" prompt, so it survives restarts and compaction. Children inherit it on their prompt admission (spawn, resume, teammates); foreground agent results, background handbacks and Workflow summaries hand a tainted child's record back. In a tainted run, egress (webfetch, websearch, MCP calls, cross-machine messages, network commands, unparseable or inline-shell commands), external writes and secret-path reads turn every allow into ask: config, Session and saved rules, Mode grants, hook allows and auto always_allow. allow_when_tainted keeps its allow; denies and ceilings are unchanged. dont-ask and unattended asks deny as not pre-approved; bypass runs as before unless bypass is "deny". A classifier allow still asks for risky actions and non-read-only commands, with its reason in the prompt; permission.asked carries metadata.taint and an "always" reply does not exempt later tainted calls. The auto classifier gets the taint evidence and a trusted system line, and permission.auto_decided records tainted. Add permissions.untrusted_content settings (taint, bypass, scan, allow_when_tainted) with strict validation; project layers and project-defined profiles may only tighten them, with one warning per file and key.
…narios and docs Add the untrusted-content topic to the base prompts (version 3), a labeled injection/benign fixture set with a recall and false-positive gate, scripted end-to-end scenarios linked from the recovery manifest, and document the frame, signals, taint and settings. Narrow the email transfer verb so sign-in form labels are not secret_access hits, and update the Copilot test marker for the framed summarizer prompt.
- Keep lower layers' permissions.untrusted_content when a project layer replaces permissions with a string, array or null. - Egress: wrappers (timeout, busybox, ...), non-literal program words, git value options, publish after options, PowerShell backslash paths and quoting, collab writes, computer.control, MCP resource reads. - Secret reads: symlinks, shell globs, $env:USERPROFILE, grep matches inside secret files in a tainted run. - @server:uri and template @ references carry their untrusted record on the prompt admission; a user prompt clears taint, then notes it. - "always" no longer cascades onto pending taint asks. - history_search, shell reads of overflow files and workflow direct tools.read/grep are framed and recorded. - Detector v2: agent-aimed command requests, transfer destinations, data-named path placeholders, companion pipe-to-shell rules, wider invisible-character set, char-offset gaps; benign install/API samples. - Sessions report the effective notice policy; TUI and dashboard use it and share a notice fixture. - Typed taint decision source, authorize_as split, Windows CI filters.
The bash half of the scan-all test starts a real shell, which the Windows runner cannot spawn (msys NtCreateDirectoryObject denied); it now lives in its own #[cfg(unix)] test like the file's other shell test. read_sse_until returned as soon as the needle appeared, so a chunk that ended mid-line left a truncated data line for the context-meter test to parse. It now waits for the blank line ending the needle's event.
…ut.redact settings Move the credential detectors from memory/secrets.rs into cyber_core::redact and give each a kind. Add provider formats (AWS, GitHub, OpenAI, Anthropic, Slack, Stripe live, Google, GitLab, npm, JWT, full PEM blocks), Authorization values, URL passwords, guarded secret assignments and exact matching of known secret values. Matches become [REDACTED:<kind>] and keep their line breaks; existing markers are never re-matched. Memory, git text, telemetry and exports keep their own marker and the permissive assignment filter through the same detector list; only the value of an assignment is replaced now. Validate tool_output.redact (enabled, patterns, allow) with bounded regexes. Project and plugin layers may only add patterns: their enabled: false and allow entries are dropped with a warning, and patterns concatenate across layers. Optimize the regex crates in dev builds so large results stay fast.
…n markers in writes
Budget::truncate redacts the raw result first, then measures and cuts the
redacted text; the overflow file keeps the raw text. read redacts the whole
file before paging. The host builds the Location's redactor from the settings,
secret-named environment variables, secret config entries and resolved
provider keys, and records what each call hid.
The runtime redacts the settled output again and stores that text, so the
model and session.tool.settled never differ; the event and CallState carry
redactions {count, kinds}. Job completion notices, workflow summaries and
session imports are redacted too.
write, edit, apply_patch and notebook_edit refuse new text holding a marker
the file does not already hold, and shell and PTY input refuse markers, so a
placeholder never replaces a real secret. The base prompt (version 3) explains
the marker.
read_sse_until returned as soon as the needle appeared, so a chunk that ended mid-line left a truncated data line for the context-meter test to parse. The longer base prompt moved the chunk boundary into that line. Wait for the blank line that ends the needle's event.
The TUI adds a line under the tool card, the dashboard an amber badge in the tool summary, and exec adds redactions to the stream-json tool_result line and a stderr note in text mode.
OpenSpec change add-tool-output-redaction (tool-registry, builtin-tools, configuration, system-context, provider-credentials, tui, web-client, exec-mode), the local-core guide section, the feature row and the roadmap entry.
…iles The two shell-driven redaction tests are #[cfg(unix)], but their helpers, constants and imports were not, so clippy -D warnings failed on Windows with dead-code and unused-import errors.
This was referenced Oct 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Secrets in tool results are hidden before provider requests and stored settlements. The same policy covers auxiliary webfetch summaries and the untrusted-content evidence later reused by the permission classifier.
Behavior
tool_output.redactdefaults on. User settings may configure patterns/allow entries; project/plugin layers may only tighten it. The combined base prompt version is 4 and retains both redaction and untrusted-content guidance.OpenSpec:
add-tool-output-redaction. This head includes #78/#79 and merged #81 to verify composition. #78 and #79 have merged.Verification
Local checks at 013c63b pass: 317 core tests, 85 configuration tests, 42 tool integration tests, two runtime settlement-redaction tests, three webfetch rendering/summary unit tests, the CLI dual-metadata regression, 52 web tests/typecheck, scoped server/tools Clippy, Rust formatting, current OpenAPI/SDK generation, and all 114 strict OpenSpec items. Spec lint reports zero errors and 25 warnings. Before-fix executions reproduced overlapping-secret suffix exposure, raw secrets in injection metadata, and raw page/origin values in a real HTTP/scripted-summary request. Hosted all-target checking then found an invalid
redactionsmember in the queue-head test fixture. Revision df22698 removes that member; both queue-ordering tests and formatting pass locally. Revision 583558a then separates the summary-provider privacy assertions into a focused helper, resolving the CI complexity failure without removing assertions. Full workspace/all-targets Clippy with warnings denied and all 20 web-skills integration tests pass locally. The Linux full suite at 583558a ran 5,502 tests: 5,501 passed, one failed, and 21 configured cases were skipped. The single failure was the MCP redaction regression expecting unframed output after composition with the MCP provenance contract. Revision 158ec84 requires the complete redacted provenance frame in both the model request and stored settlement; its exact real-MCP regression passes locally. All 38 current-head hosted checks now pass. The prospective merge tree equals the tested head tree.Limits