Skip to content

fix(hermes): filter private transcript fields - #683

Open
xnne-bot wants to merge 2 commits into
NevaMind-AI:mainfrom
XnneHangLab-Mirror:fix/hermes-transcript-private-fields
Open

xnne-bot wants to merge 2 commits into
NevaMind-AI:mainfrom
XnneHangLab-Mirror:fix/hermes-transcript-private-fields

Conversation

@xnne-bot

@xnne-bot xnne-bot commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

📝 Pull Request Summary

Remove known Hermes runtime/private metadata at the prepared-transcript output boundary while preserving its OpenAI-shaped message and tool semantics.


✅ What does this PR do?

  • Adds a delete-only Hermes sanitizer for the projected SQLite row shape.
  • Removes the row-level timestamp and provider replay fields call_id, response_item_id, and extra_content from serialized tool_calls.
  • Preserves canonical tool-call id, function names/arguments, tool-result tool_call_id, message content, and unknown future fields.
  • Keeps the serialized tool_calls container shape unchanged and safely leaves malformed or unsupported values untouched.
  • Leaves SQLite reads, classification, source data, timestamps used by the cursor, and prepared row ordering unchanged.
  • Adds focused sanitizer coverage and an end-to-end prepare_transcripts() regression.

🤔 Why is this change needed?

Hermes prepared transcripts currently retain per-row timestamps and provider-specific replay metadata that are needed by the host runtime but not by memory or skill mining. Hermes already projects its SQLite rows to a narrow OpenAI-shaped schema, so this change filters only the remaining known runtime fields rather than widening or rebuilding that projection.

A read-only validation over the latest 10 local Hermes sessions kept all 202 prepared rows and all 74 tool call/result pairs in the same order while reducing combined output from 345,642 bytes to 324,554 bytes (6.1%). The source database bytes and staged cursor timestamps remained unchanged.

This follows the delete-only, unknown-field-preserving sanitizer contract established by #677, #680, and #681.

MrXnneHang and others added 2 commits September 2, 2026 17:34
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@xnne-bot

xnne-bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

基于本机真实 Hermes state.db 的复核结果:这个 PR 值得保留。

Hermes 与已关闭的 Cursor PR 不同:adapter 直接从 state.db.messages 读取,再由 memU 投影为 prepared transcript,因此本 PR 的 delete-only sanitizer 正好处于 memU 可控的输出边界。

实际数据库统计:

  • messages:1,280 行,全部带 timestamp。
  • tool_calls:433 个 JSON 容器、669 个 tool call。
  • 669 个 tool call 全部含有 id、call_id、response_item_id、type、function。
  • 本 PR 删除 call_id、response_item_id,但保留 tool call 的 id,并保留 tool result 的 tool_call_id,因此 tool call/result 的语义关联仍在。
  • 当前样本中 extra_content 出现次数为 0;保留它在 delete list 中仍是合理的已知 provider replay 字段处理,且未知字段保持不变。

我复制同一份真实 DB,在 main 与本 PR 上分别实际执行 prepare_transcripts(最近 10 个 session)。source DB 未被修改、两个 pending cursor 完全一致,且 prepared row 的数量与顺序不变:

输出 main 本 PR 缩减 行数
memory transcript 总计 64,660 B 56,563 B 12.5% 58 → 58
full transcript 总计 281,184 B 268,193 B 4.6% 144 → 144
合计 345,844 B 324,756 B 6.1% 202 → 202

另外在 PR branch 执行了:

uv run python -m pytest tests/test_host_sessions.py -k hermes
# 8 passed

因此建议合入:这是有真实字段命中、可量化收益、且保留未知字段的窄范围 delete-only filter。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants