knowledge: improve review precision from BCApps PR 10466 feedback

[Wenjie Fan (gggdttt)]

Summary

Improves BCQuality knowledge based on maintainer thumbs-down feedback from explicit BCApps PR review runs.

Source feedback

• Resolve purchase lines to items on exact description match BCApps#10466 (comment)

Validation

• Frontmatter and article structure validation completed.

Generated by the BC-ALAgentsInternal self-improvement workflow.

Offline evaluation: regression

Candidate correctness failed: unexpected or missed findings remain. Existing ignored gold comments retain their neutral scoring semantics.

Preparation attempt 1: validated.

Selection: new. New: synthetic__breaking-fallback-description-match-01. Reused (complete payloads): ``.

• synthetic__breaking-fallback-description-match-01 / calibration_context / Resolve purchase lines to items on exact description match BCApps#10466 (comment): Negative (THUMBS_DOWN) reaction with a developer reply ('That's new behavior this PR is doing') that confirms rather than refutes the described behavior change, so this is a recommendation disagreement (opt-in gating) over a genuine finding, not a clean negative. Keeps an expected finding on the fallback resolution call site so the boundary isn't miscoded as a false-positive guard. Limitation: does not validate the reviewer's specific 'add an opt-in setting' remediation, only the underlying silent-default-change premise; severity/remediation choice remains a human-reviewed judgment call.
  Coverage references and outcome shapes are validated mechanically. Semantic equivalence, severity calibration and recommendation quality are NOT proved by these checks or a matching F1.

Common engine code: ecf8e31759d6ddd6d78e3a0b7836b40134368009; dataset SHA256: AB3371B46C6015755BEC8562BDEDEF179EB6385D8D51C5C9CDAD8BA6D19A4AF7.
Model: gpt-5.6-luna; judge: gpt-5.3-codex. Exact D: synthetic__breaking-fallback-description-match-01.
Dataset base: 26365d6a41a96fac41df9dcc8339cb71241fcb1f; candidate: 273d0e0a458c7d59771272e6a54036340fd6b122.

Arm	Evaluation commit	Engine pin	Knowledge pin
baseline	f400d204bf717089cab86e3a9e38c9cb7110d2e4	4407f20b44fe92e3693d0444d25cc3430f1d5262	8584217c7506eea7eef27a9db353d78c0cd6a9a1
candidate	d38b7b8cddf87e6389d82b111d73b02e6c90e4bd	b03ac0db52874297405e05b70324bcc74da6ddcd	c08f36de18453cffe1830e0d28fa68ce101f8b77

Baseline

Run: https://github.com/microsoft/BC-Bench/actions/runs/34346037868; conclusion: success; wall clock: 6.6 minutes.
Raw findings and runtime pin proofs are retained in the self-improvement-feedback artifact and the linked evaluation run.

Entry	Expected	Generated	Missed	Unexpected	F1	AI credits	Agent seconds
synthetic__breaking-fallback-description-match-01	1	0	1	0	0	unavailable	172.605483866

Observed AI credits: unavailable; coverage 0/1 entries. Missing entries are not extrapolated into a total.

Aggregate metric	Value
total	1
expected_comment_count	1
generated_comment_count	0
matched_comment_count	0
missed_comment_count	1
incorrect_comment_count	0
precision	1
recall	0
f1	0

Candidate

Run: https://github.com/microsoft/BC-Bench/actions/runs/34346655887; conclusion: success; wall clock: 24.5 minutes.
Raw findings and runtime pin proofs are retained in the self-improvement-feedback artifact and the linked evaluation run.

Entry	Expected	Generated	Missed	Unexpected	F1	AI credits	Agent seconds
synthetic__breaking-fallback-description-match-01	1	2	1	2	0	unavailable	1276.604446395

Observed AI credits: unavailable; coverage 0/1 entries. Missing entries are not extrapolated into a total.

Aggregate metric	Value
total	1
expected_comment_count	1
generated_comment_count	2
matched_comment_count	0
missed_comment_count	1
incorrect_comment_count	2
precision	0
recall	0
f1	0

Missing telemetry is unavailable, not zero. Evaluation-only metrics exclude candidate generation and are not the full-cycle cost.
Human review must verify source-patch fidelity, gold correctness, and target attribution. No automatic merge or branch-protection claim is made.

[Jesper Schulz-Wedde (JesperSchulz)]

Requesting changes on c08f36de18453cffe1830e0d28fa68ce101f8b77:

• The blanket exclusion is not supported by the source thread. “That’s new behavior this PR is doing” confirms intent, but intent/PR description alone does not prove a default runtime change has no compatibility impact. Narrow this to reject speculative findings unless they identify a concrete shipped contract, setup, extension point, or workflow dependency; do not declare all new matching/resolution behavior outside review.
• The gate itself reports candidate correctness failure: F1 0, the retained expected finding missed, and two unexpected findings. The new article is also absent from the breaking-changes skill’s supposedly exhaustive targeted checks, and no paired AL fixture exercises it, so the green exact-tree validators do not test this rule. Please wire deterministic routing and provide passing semantic evaluation/fixture coverage for the narrowed boundary.