feat: contradiction benchmark segment + uteke supersede CLI (#1172 F3) - #1192
Merged
Conversation
Fase 3 of #1172 closes the epic with an active-store knowledge-update segment proving conflict resolution end-to-end: - contradiction_segment.py: 40 topics x (stale + winner + 3 distractors), semantic 'which X does topic use now?' queries; baseline measured for ALL strategies on the unresolved store, then one resolution pass, then resolved metrics (code-scanning fixes: topic->thing map instead of a broken next() query that malformed questions; baseline-ordering so later strategies never measure a resolved store) - Results (RESULTS.md + results_contradiction_f3/metrics.json): - fusion (default): winner@1 0.850 -> 1.000, MRR 0.925 -> 1.000, stale@5 1.000 -> 0.000 — unresolved conflicts pollute top-5 on every strategy; supersede clears them from the retrieval surface - hybrid baseline ranks the stale fact top-1 on 97.5% of topics (lexical match of 'uses X' with 'use now?'); resolved stale@1 = 0 - ledger listed all 40 resolutions (F2 surface verified in-loop) - uteke supersede <old> <new> [--reason]: CLI surface parity for supersession (was MCP/HTTP only) — prefix-aware, --json, points to contradictions undo for restore
🔍 Cora AI Code Review✅ No issues found. Code looks good! Review powered by cora-code · BYOK · MIT |
This was referenced Sep 6, 2026
Closed
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fase 3 of #1172 — closes the epic with an active-store benchmark segment proving conflict resolution end-to-end, plus the missing CLI surface:
benchmarks/longmemeval/contradiction_segment.py— 40 topics × (stale fact + winner fact + 3 distractors); queries ask "which {thing} does {topic} use now?" (semantic, no keyword echo). Baseline metrics are measured for ALL strategies on the unresolved store first, then one supersede pass resolves everything, then resolved metrics. Harness + raw metrics committed (results_contradiction_f3/metrics.json); findings published inbenchmarks/longmemeval/RESULTS.md.uteke supersede <old> <new> [--reason]— new CLI command, parity with the MCP tool and HTTP surface (supersession was previously unreachable from the CLI). Prefix-aware,--json, human output points atcontradictions undofor restore.uteke supersede), CHANGELOG, RESULTS.md section.Why
#1172 F3. F1/F2 built provenance and the resolution ledger; F3 demonstrates the payoff on an active store: ranking alone often picks the winner (85–95% top-1), but only explicit resolution guarantees stale facts leave the retrieval surface entirely — the difference between "usually right" and deterministic freshness for agent memory. The segment also exposed and closed a real surface gap (no CLI supersede).
Testing
cargo test --workspace— 667 passed, 0 failed; fmt clean; clippy 0 warnings;cora review --stagedpass (2 findings from its first pass fixed: broken topic→thing query mapping; per-strategy baseline ordering that would measure a resolved store)uteke supersedeexercised live in the segment loop (40 resolutions via the new command)