RLE bench design — N × ticks for harness QA → stats - #73
Merged
Conversation
Lock the four-stage bench path (QA → σ pilot → horizon → sized compare) so N=1 seed-42 cells are not treated as significance. Fold 2026-09-07 gate scores; 10-tick stays software validation first. Co-authored-by: Jason <jkbennitt@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Docs-only memo for how we get from a working harness to a number we can defend. No code, deps, defaults, or matrix runs. Do not merge until reviewed.
What
Adds
docs/bench-design-n-ticks.mdplus a one-line pointer from the README live-board paragraph.Locked path (do not conflate)
n ≈ 16 σ²/δ²per cell (80% power, α=0.05); prefer paired same-seed setsCurrent scoring-1.2 board is almost all seed-42 N=1: no within-cell σ. Significance without σ is theater. Flash cells ~$0.02–$0.90 — do not incinerate a full matrix before σ + horizon.
Gate fold (2026-09-07)
f818246e@41fd015; ~$0.025 consoleOut of scope
No CLI/default changes. No matrix. No XAI.