The README says, under "Three details worth getting right":
Score can be fractional. Its value is the probability-weighted position across ordered rubric levels. Inspect the distribution; the same mean can conceal different uncertainty.
The reports do not help a reader do that. src/contract.mjs#L40 checks that a Score answer's mean matches its probabilities, but the report and the CLI output surface only the mean.
Add a small, deterministic spread measure for every Score answer and include it in the JSON report and the human-readable line the CLI prints for a case. Standard deviation of the level index under probabilities is the simplest option and needs no dependency. Normalised entropy is also fine; pick one, name it clearly, and document it in one sentence in docs/evaluation.md.
Add a test in test/contract.test.mjs or test/examples.test.mjs with two authored distributions that share a mean and differ in spread, asserting the two spread values differ in the expected direction.
Done when
- Score answers in reports carry the new field.
- The CLI prints it next to the Score value for example 07.
- The test above passes and
npm run check passes.
How to work on this
- Fork, then
git clone and run npm run check (Node 20.17+, no install step, no API key needed).
- Make the change, run
npm run check and npm run docs (CI fails if a generated example README is stale).
- Open one focused PR. Say what you verified, and mention any substantial AI assistance (see CONTRIBUTING.md).
Comment here if you want to take it, and ask anything that is unclear.
The README says, under "Three details worth getting right":
The reports do not help a reader do that.
src/contract.mjs#L40checks that a Score answer's mean matches itsprobabilities, but the report and the CLI output surface only the mean.Add a small, deterministic spread measure for every Score answer and include it in the JSON report and the human-readable line the CLI prints for a case. Standard deviation of the level index under
probabilitiesis the simplest option and needs no dependency. Normalised entropy is also fine; pick one, name it clearly, and document it in one sentence indocs/evaluation.md.Add a test in
test/contract.test.mjsortest/examples.test.mjswith two authored distributions that share a mean and differ in spread, asserting the two spread values differ in the expected direction.Done when
npm run checkpasses.How to work on this
git cloneand runnpm run check(Node 20.17+, no install step, no API key needed).npm run checkandnpm run docs(CI fails if a generated example README is stale).Comment here if you want to take it, and ask anything that is unclear.