feat(phase-56): Honest Benchmarks — real recall@k, locomo10.json, HOLD comparison marketing

[RichardHightower]

Phase 56: Honest Benchmarks

Closes the v3.0 benchmark claim/reality gap from docs/plans/v3.1-make-it-true-plan.md. Sequential after Phase 55 (#33, merged).

56-01 Custom harness

• recall_at_k is labeled relevant items in top-k, not accuracy under another name (committed mock run: accuracy 0.88, recall@5 0.86)
• compression_ratio reads setup file contents, not path-string lengths
• memory add / memory search failures abort the run
• Isolated mock store per test (unit-tested; no shared-store bleed)
• 25 fixtures (temporal / multi / compress) with distractor sessions

56-02 LOCOMO adapter v2

• Parses real locomo10.json (sample_id, session_N + timestamps, qa with integer category, numeric answer)
• Invented v1 schema (conversation_id/turns/questions) is rejected
• Download script fetches GitHub data/locomo10.json and prints LICENSE.txt (CC BY-NC 4.0)
• One isolated store per conversation
• Substring mode is context_hit_rate — never labeled a LOCOMO score
• --scorer llm-judge is the only locomo_llm_judge path (temp 0, model recorded; needs API key)
• memory add --timestamp --session-id --role so a live-daemon run can keep session time

56-03 Run / record / gate

Committed artifacts from actual runs:

File	What
benchmarks/results/custom-harness-mock.json	25 tests, backend=mock, 22/25. Not production quality.
benchmarks/results/locomo-smoke.json	1 conversation, 4 questions, metric=context_hit_rate, score 0.5. Not a LOCOMO score.

CI benchmark-smoke now runs cargo test -p memory-bench and memory-bench smoke (parse → ingest → retrieve → score). --help-only is gone. Job is required.

--compare refuses mock scorer so context_hit_rate cannot share a table with MemMachine/Mem0 numbers. Competitor rows name their metric.

Decision gate

HOLD comparison marketing. No locomo_llm_judge artifact (no API key / no full locomo10.json run). Positioning (Phase 57) does not depend on a LOCOMO score.

Test plan

• cargo test -p memory-bench — 38 passed
• cargo clippy -p memory-bench -- -D warnings
• cargo fmt --check -p memory-bench -p memory-cli
• cargo run -p memory-bench -- smoke — 1 conversation, metric=context_hit_rate
• cargo run -p memory-bench -- locomo --scorer mock --compare exits non-zero

[RichardHightower]

Rebased onto main after #35 merged (rustc 1.97 pin + result_large_err allow). Also fixed the rustdoc bare-urls failure in memory-bench (locomo.rs module docs now use <https://...>).

This should unredden Clippy and Documentation. Will merge when CI is green.