FYI: your report schema's (lack of a) rollup field cited as design precedent in an unrelated RFC

[adhabnr-ux]

Hi — I maintain EvalPort, a small open JSON-Schema spec for portable LLM evaluation test cases/results. This isn't a feature request for this project, so I'm posting it in General rather than opening an issue — just a citation you might otherwise never see.

I'm working through an RFC (Discussion #45) for an optional group field so one eval ResultSet document can point at sibling result sets from the same sweep or run (mutation testing, hyperparameter sweeps, multi-model comparisons — a generic version of the same problem this project solves for one specific domain). Before proposing a shape, I read packages/report-schema/src/mutation-testing-report-schema.json end to end, since it's the closest real precedent there is to one of the RFC's motivating cases: grouping per-mutant results into one sweep. Two things from it directly shaped the proposal: the status enum (Killed/Survived/NoCoverage/CompileError/RuntimeError/Timeout/Ignored/Pending) is exactly the kind of open-but-conventional per-member outcome vocabulary the new group.role field is meant to carry, and — checked specifically — there's no mutation-score or rollup field anywhere in the schema; thresholds.high/thresholds.low are display bounds, not a computed score, so the actual mutation score is left to whatever's consuming the report. That's a deliberate design choice on your side, and it turned out to be the strongest precedent for the RFC's own decision not to standardize a rollup document — a contributor in the discussion thread called it out directly as "the right precedent."

No ask attached — I'm not proposing an EvalPort/Stryker adapter, just flagging that a specific, considered choice in your schema informed a decision elsewhere. Happy to be told if I've misread anything above.

[hugo-vrijswijk]

Hi! I am not exactly sure what your question here is. MTE's score is not in the report by design so as not to have duplicate, possibly conflicting, data in the report. All the data you need to calculate a score is in the report, and there is a standardized way to calculate the score.

If you have a genuine question or request please expand on it in your own language. Your post is a bit too much AI slop for me to read 😅

[adhabnr-ux]

Fair, and sorry for the noise. No real question here — you answered it anyway: the score being left out on purpose, with a documented way to compute it, is exactly what I wanted to confirm. Thanks for taking the time.