From 5057d1bfb68fb86d0f0c658c723260b2485622db Mon Sep 17 00:00:00 2001 From: joyful-ii-V-I Date: Tue, 22 Sep 2026 07:03:23 -0400 Subject: [PATCH 1/3] docs(readability): cite the v0.6.2-pinned 412-pair / 37.4% proxy number --readability's UNVALIDATED caveat (src/cli.h, mirrored in README.md) cited 484 pairs / 30.2% right-direction. That figure came from bench/readability_refactor_pairs.py walking `git log --all`, which sweeps every branch in this clone's shared .git, so the population moved whenever an unrelated lane was pushed and the number does not reproduce. PR #313 (9aecbc96) pins the script to the v0.6.2 tag by default; on that pinned population the result is 412 pairs, 154 right-direction, 37.4%, still worse than chance. Both surfaces now cite it and say it is pinned, so the next reader knows it cannot drift. Also: the --quality-delta help named its tenth kind `reuse-decline`, but the binary emits kind="new-clone-of-reused-helper" (src/quality.h; the same name the verb's own legend and docs/EVALS.md list). They are one kind under two names. The help now names the emitted string, and the EVALS.md note on abbreviations says `reuse-decline` is an older name for it. docs/COMMANDS.md regenerated. Co-Authored-By: Claude Opus 5 --- README.md | 7 ++++--- docs/COMMANDS.md | 4 ++-- docs/EVALS.md | 4 ++-- src/cli.h | 11 ++++++----- 4 files changed, 14 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index fdb1e5801..b79e9c00c 100644 --- a/README.md +++ b/README.md @@ -1161,10 +1161,11 @@ rather than blurring it: Siegmund, ICSE 2021, [doi:10.1109/ICSE43902.2021.00056](https://doi.org/10.1109/ICSE43902.2021.00056)) — `--readability` emits volume and stops there; difficulty and effort are computed nowhere in this tree. **Unvalidated against human judgement, stated plainly rather than assumed:** this is a - deterministic ordering signal, not a checked one. Our own proxy measurement — 484 matched + deterministic ordering signal, not a checked one. Our own proxy measurement — 412 matched before/after function pairs mined from 80 refactor/simplify/cleanup commits in this repository's own - history — found the lens agrees with the commit's implied readability direction on only 30.2% of - pairs, worse than chance. That is a construct-validity finding about the ordering claim, not a bug in + history, pinned to the `v0.6.2` tag rather than `git log --all` so the population cannot drift when an + unrelated branch is pushed — found the lens agrees with the commit's implied readability direction on + only 37.4% of pairs (154 of 412), worse than chance. That is a construct-validity finding about the ordering claim, not a bug in the arithmetic (a separate self-consistency check confirms the formula computes exactly what it says it computes); the lens itself is unchanged pending a proper human study, and `--help=--readability` carries the same caveat where a CLI reader meets it. diff --git a/docs/COMMANDS.md b/docs/COMMANDS.md index 8e491b610..8f5136dc0 100644 --- a/docs/COMMANDS.md +++ b/docs/COMMANDS.md @@ -1839,7 +1839,7 @@ $ ./build/ripwire . --clones **Answers:** rank functions least-readable first, by volume, token entropy and length per-function readability lens, LEAST readable first: vol= Halstead volume V (N*log2(eta)), ent= Shannon token entropy E, lines= L, posnett= sigmoid(8.87 - 0.033V + 0.40L - 1.5E) (Posnett/Hindle/Devanbu, MSR 2011). -APPROXIMATION, disclosed: ONE token-class table serves every language (keywords + punctuation = operators, identifiers + literals = operands), with no per-grammar refinement, so V is cross-language and not a per-grammar Halstead count. The formula was fitted on snippets of 20 lines or fewer, so it is a RANKING lens, not a grade: read the ORDER of the rows, not the number on any one of them. Pages with limit=N (offset=M); default 40 rows. Declarations with no body are not measured. UNVALIDATED (t14-cleanup #8): this is a deterministic ORDERING signal that has not been checked against human judgement of readability. Our own proxy measurement — 484 matched before/after function pairs from 80 refactor/simplify/cleanup commits in this repo's own history — found the lens agrees with the commit's implied readability direction on only 30.2% of pairs, which is worse than chance and suggests the ranking may run backwards more often than not. Treated here as a signal to weigh, never a verdict; do not read a low posnett= as proof a function needs work. +APPROXIMATION, disclosed: ONE token-class table serves every language (keywords + punctuation = operators, identifiers + literals = operands), with no per-grammar refinement, so V is cross-language and not a per-grammar Halstead count. The formula was fitted on snippets of 20 lines or fewer, so it is a RANKING lens, not a grade: read the ORDER of the rows, not the number on any one of them. Pages with limit=N (offset=M); default 40 rows. Declarations with no body are not measured. UNVALIDATED (t14-cleanup #8): this is a deterministic ORDERING signal that has not been checked against human judgement of readability. Our own proxy measurement — 412 matched before/after function pairs from 80 refactor/simplify/cleanup commits in this repo's own history — found the lens agrees with the commit's implied readability direction on only 37.4% of pairs (154 of 412), which is worse than chance and suggests the ranking may run backwards more often than not. That population is pinned to the v0.6.2 tag, never `git log --all`, so it cannot drift when an unrelated branch is pushed. Treated here as a signal to weigh, never a verdict; do not read a low posnett= as proof a function needs work. **Try it** @@ -2831,7 +2831,7 @@ $ ./build/ripwire . --quality-baseline --allow-dirty ### `--quality-delta` -**Answers:** before a PR: report ONLY what your change made worse, across 10 kinds agent self-check before a PR (pair with --test-gate): report ONLY what a change made worse vs the baseline (10 kinds: complexity/verbosity/nesting/params/dup/dead/api-surface + error-masking/short-horizon-churn/reuse-decline); +**Answers:** before a PR: report ONLY what your change made worse, across 10 kinds agent self-check before a PR (pair with --test-gate): report ONLY what a change made worse vs the baseline (10 kinds: complexity/verbosity/nesting/params/dup/dead/api-surface + error-masking/short-horizon-churn/new-clone-of-reused-helper); every finding is classified by ORIGIN: a symbol that EXISTED at the baseline and got worse (preexisting-worse="N", no attribute on the row) vs one that exists only because the code is NEW (new-symbol="N", origin="new-symbol" on the row). A small numeric delta is additionally sev="minor". EXIT 2 ONLY on preexisting-worse AND major AND unacked — the gating="N" header count. New-symbol rows are still PRINTED (they are the debt you are adding — read them), they just never gate; exit 0 means "nothing that already existed got worse", not "clean". Clone kinds classify by member set (new-symbol only if EVERY member is new); short-horizon-churn is preexisting by construction. LIMIT: origin is canonId (path::scope::name) identity, so a RENAMED/MOVED symbol reads as new and a regression carried in with the move will not gate. Test-fixture dirs + doc sections are exempt from dead-code/churn; churn needs COMMITTED thrash evidence (rewritten across recent commits AND again by this diff), never the current edit alone WHICH FLOOR IT COMPARES AGAINST, and a side effect: the sidecar is honored only when the sha it was pinned at EQUALS the current git HEAD (strict equality — an ancestor commit describes a DIFFERENT tree, so everything committed since would read as your regression). A sidecar pinned anywhere else is STALE: this verb then DELETES it from your working tree (self-heal, so the next run does not rediscover the dead pin) and auto-compares the working tree vs git HEAD instead. Re-pin with --quality-baseline. The read-only MCP quality_delta verb applies the SAME staleness test but never deletes. A sidecar at the current HEAD that ANOTHER ripwire build pinned (its producer stamp names other sources — a dead set depends on how calls were resolved) is FOREIGN: both arms ignore it, never delete it, and auto-compare vs git HEAD. Which floor was actually used is on every report as baseline=: sidecar | git-HEAD | git-HEAD (stale sidecar removed) | git-HEAD (stale sidecar ignored) | git-HEAD (foreign sidecar ignored) — the stale two say a stale sidecar existed, and 'removed' means the file is gone. A non-git root has no HEAD to fall back to, so its sidecar is honored whenever this build pinned it; without one there, or with another build's, the verb exits 1. diff --git a/docs/EVALS.md b/docs/EVALS.md index 97eff9a02..8deab027e 100644 --- a/docs/EVALS.md +++ b/docs/EVALS.md @@ -6145,8 +6145,8 @@ These are the exact `kind=` strings the binary emits, from `src/quality.h`: `complexity` · `verbosity` · `nesting` · `params` · `duplication` · `dead-code` · `api-surface` · `error-masking` · `short-horizon-churn` · `new-clone-of-reused-helper` -Note that some user-facing summaries abbreviate four of these (`dup`, `dead`, `churn`, -`clone-of-reused-helper` / `reuse-decline`). **Match against the strings above** when grepping real +Note that some user-facing summaries abbreviate three of these (`dup`, `dead`, `churn`), and older +text calls the last one `clone-of-reused-helper` or `reuse-decline`. **Match against the strings above** when grepping real output. The verb reports only what a change made *worse*, against git HEAD. `--quality-ack` records a diff --git a/src/cli.h b/src/cli.h index 327a61c55..272edceb8 100644 --- a/src/cli.h +++ b/src/cli.h @@ -1388,12 +1388,13 @@ inline constexpr char kHelpHead[] = " grade: read the ORDER of the rows, not the number on any one of them. Pages with limit=N\n" " (offset=M); default 40 rows. Declarations with no body are not measured.\n" " UNVALIDATED (t14-cleanup #8): this is a deterministic ORDERING signal that has not been\n" - " checked against human judgement of readability. Our own proxy measurement — 484 matched\n" + " checked against human judgement of readability. Our own proxy measurement — 412 matched\n" " before/after function pairs from 80 refactor/simplify/cleanup commits in this repo's own\n" " history — found the lens agrees with the commit's implied readability direction on only\n" - " 30.2% of pairs, which is worse than chance and suggests the ranking may run backwards more\n" - " often than not. Treated here as a signal to weigh, never a verdict; do not read a low\n" - " posnett= as proof a function needs work.\n" + " 37.4% of pairs (154 of 412), which is worse than chance and suggests the ranking may run\n" + " backwards more often than not. That population is pinned to the v0.6.2 tag, never\n" + " `git log --all`, so it cannot drift when an unrelated branch is pushed. Treated here as a\n" + " signal to weigh, never a verdict; do not read a low posnett= as proof a function needs work.\n" " --nonlocal-state per function, the non-local mutable state it can reach, most writes first\n" " per function, the NON-LOCAL MUTABLE STATE it can reach, MOST WRITES FIRST: writes= reads= are the\n" " distinct cells this function OR its transitive callees write / read; direct_writes= direct_reads=\n" @@ -1636,7 +1637,7 @@ inline constexpr char kHelpHead[] = " later --quality-delta against it carries baseline_absorbed=\"N\" — so a green exit beside that attribute reads as\n" " \"clean SINCE THE PIN\", never \"clean\". Refused alone.\n" " --quality-delta before a PR: report ONLY what your change made worse, across 10 kinds\n" - " agent self-check before a PR (pair with --test-gate): report ONLY what a change made worse vs the baseline (10 kinds: complexity/verbosity/nesting/params/dup/dead/api-surface + error-masking/short-horizon-churn/reuse-decline);\n" + " agent self-check before a PR (pair with --test-gate): report ONLY what a change made worse vs the baseline (10 kinds: complexity/verbosity/nesting/params/dup/dead/api-surface + error-masking/short-horizon-churn/new-clone-of-reused-helper);\n" " every finding is classified by ORIGIN: a symbol that EXISTED at the baseline and got worse (preexisting-worse=\"N\", no attribute on the row) vs one that exists only\n" " because the code is NEW (new-symbol=\"N\", origin=\"new-symbol\" on the row). A small numeric delta is additionally sev=\"minor\". EXIT 2 ONLY on preexisting-worse AND\n" " major AND unacked — the gating=\"N\" header count. New-symbol rows are still PRINTED (they are the debt you are adding — read them), they just never gate; exit 0 means\n" From 83bfa2e1e3d105687492388955b7cf01641de968 Mon Sep 17 00:00:00 2001 From: joyful-ii-V-I Date: Tue, 22 Sep 2026 07:03:50 -0400 Subject: [PATCH 2/3] docs(changelog): Unreleased entry for the pinned --readability number Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 96e7bedef..419d40114 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,22 @@ not published here — see `docs/EVALS.md` for the instruments behind the headli --- +## [Unreleased] + +### Fixed — `--readability`'s caveat cited a proxy number that did not reproduce + +The `UNVALIDATED` note in `--help=--readability` and the README said the lens agrees with a refactor +commit's implied readability direction on 30.2% of 484 function pairs. That population came from +`bench/readability_refactor_pairs.py` walking `git log --all`, which reads every branch in the clone's +shared `.git`, so the count changed whenever an unrelated branch was pushed. With the script pinned to +the `v0.6.2` tag (#313), the same method (80 refactor/simplify/cleanup commits, ≤400 changed lines each) +gives **154 of 412 pairs, 37.4%**. That is still worse than chance, so the caveat's conclusion holds. +Both surfaces now cite the pinned figure and name the tag. The `--quality-delta` help also named its +tenth kind `reuse-decline`; it now uses the `kind=` string the binary emits, +`new-clone-of-reused-helper`. + +--- + ## [0.6.2] — 2026-09-21 ### Added — Microsoft's `cl.exe` builds the tree, so both Windows front ends compile and both gate From 322f979a94664792bc664eaa79fc469c7fb0179c Mon Sep 17 00:00:00 2001 From: joyful-ii-V-I Date: Tue, 22 Sep 2026 07:11:49 -0400 Subject: [PATCH 3/3] docs(readability): drop the flag-shaped `--all` token from the caveat; re-pin help_all readmedriftcheck (B) counts every distinct --long token in --help, so the phrase `git log --all` read as a 184th flag. Reworded in cli.h and README. printffmtparitycheck: red set was exactly {help_all} (a deliberate help edit, not a conversion); that one manifest row re-pinned. Co-Authored-By: Claude Opus 5 --- README.md | 2 +- docs/COMMANDS.md | 2 +- src/cli.h | 6 +++--- test/printf_parity.manifest | 2 +- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index b79e9c00c..c8c433fd5 100644 --- a/README.md +++ b/README.md @@ -1163,7 +1163,7 @@ rather than blurring it: tree. **Unvalidated against human judgement, stated plainly rather than assumed:** this is a deterministic ordering signal, not a checked one. Our own proxy measurement — 412 matched before/after function pairs mined from 80 refactor/simplify/cleanup commits in this repository's own - history, pinned to the `v0.6.2` tag rather than `git log --all` so the population cannot drift when an + history, pinned to the `v0.6.2` tag rather than a walk of every branch in the clone, so the population cannot drift when an unrelated branch is pushed — found the lens agrees with the commit's implied readability direction on only 37.4% of pairs (154 of 412), worse than chance. That is a construct-validity finding about the ordering claim, not a bug in the arithmetic (a separate self-consistency check confirms the formula computes exactly what it says diff --git a/docs/COMMANDS.md b/docs/COMMANDS.md index 8f5136dc0..b53867b7f 100644 --- a/docs/COMMANDS.md +++ b/docs/COMMANDS.md @@ -1839,7 +1839,7 @@ $ ./build/ripwire . --clones **Answers:** rank functions least-readable first, by volume, token entropy and length per-function readability lens, LEAST readable first: vol= Halstead volume V (N*log2(eta)), ent= Shannon token entropy E, lines= L, posnett= sigmoid(8.87 - 0.033V + 0.40L - 1.5E) (Posnett/Hindle/Devanbu, MSR 2011). -APPROXIMATION, disclosed: ONE token-class table serves every language (keywords + punctuation = operators, identifiers + literals = operands), with no per-grammar refinement, so V is cross-language and not a per-grammar Halstead count. The formula was fitted on snippets of 20 lines or fewer, so it is a RANKING lens, not a grade: read the ORDER of the rows, not the number on any one of them. Pages with limit=N (offset=M); default 40 rows. Declarations with no body are not measured. UNVALIDATED (t14-cleanup #8): this is a deterministic ORDERING signal that has not been checked against human judgement of readability. Our own proxy measurement — 412 matched before/after function pairs from 80 refactor/simplify/cleanup commits in this repo's own history — found the lens agrees with the commit's implied readability direction on only 37.4% of pairs (154 of 412), which is worse than chance and suggests the ranking may run backwards more often than not. That population is pinned to the v0.6.2 tag, never `git log --all`, so it cannot drift when an unrelated branch is pushed. Treated here as a signal to weigh, never a verdict; do not read a low posnett= as proof a function needs work. +APPROXIMATION, disclosed: ONE token-class table serves every language (keywords + punctuation = operators, identifiers + literals = operands), with no per-grammar refinement, so V is cross-language and not a per-grammar Halstead count. The formula was fitted on snippets of 20 lines or fewer, so it is a RANKING lens, not a grade: read the ORDER of the rows, not the number on any one of them. Pages with limit=N (offset=M); default 40 rows. Declarations with no body are not measured. UNVALIDATED (t14-cleanup #8): this is a deterministic ORDERING signal that has not been checked against human judgement of readability. Our own proxy measurement — 412 matched before/after function pairs from 80 refactor/simplify/cleanup commits in this repo's own history — found the lens agrees with the commit's implied readability direction on only 37.4% of pairs (154 of 412), which is worse than chance and suggests the ranking may run backwards more often than not. That population is pinned to the v0.6.2 tag, never a walk of every branch in the clone, so it cannot drift when an unrelated branch is pushed. Treated here as a signal to weigh, never a verdict; do not read a low posnett= as proof a function needs work. **Try it** diff --git a/src/cli.h b/src/cli.h index 272edceb8..1da728cb9 100644 --- a/src/cli.h +++ b/src/cli.h @@ -1392,9 +1392,9 @@ inline constexpr char kHelpHead[] = " before/after function pairs from 80 refactor/simplify/cleanup commits in this repo's own\n" " history — found the lens agrees with the commit's implied readability direction on only\n" " 37.4% of pairs (154 of 412), which is worse than chance and suggests the ranking may run\n" - " backwards more often than not. That population is pinned to the v0.6.2 tag, never\n" - " `git log --all`, so it cannot drift when an unrelated branch is pushed. Treated here as a\n" - " signal to weigh, never a verdict; do not read a low posnett= as proof a function needs work.\n" + " backwards more often than not. That population is pinned to the v0.6.2 tag, never a walk\n" + " of every branch in the clone, so it cannot drift when an unrelated branch is pushed. Treated here\n" + " as a signal to weigh, never a verdict; do not read a low posnett= as proof a function needs work.\n" " --nonlocal-state per function, the non-local mutable state it can reach, most writes first\n" " per function, the NON-LOCAL MUTABLE STATE it can reach, MOST WRITES FIRST: writes= reads= are the\n" " distinct cells this function OR its transitive callees write / read; direct_writes= direct_reads=\n" diff --git a/test/printf_parity.manifest b/test/printf_parity.manifest index 2845e8814..81180b11e 100644 --- a/test/printf_parity.manifest +++ b/test/printf_parity.manifest @@ -38,5 +38,5 @@ safe_delete 0 8b99d83566b89332ac4b2967abbb39284db775bbed601628b97b8c7909f04ac2 e verify_layer 1 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 5288c345d7d6e335f88b9c1daa8935db22e1dcf89c0c8bc1f6140d4cb5af0b48 graph_query 1 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 bfa4319feb9dee09cfbd3991cf6fbd752297e99d14de9e75768820d2a9c8832f callers_limit 0 e40ee1ddba0fbf548c58c98b91fe052af8c34f8913edb3edd85245b12a6967b9 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 -help_all 0 8f0f9dca5199beebdf9e73d578076812ecfa3de8ed6450db3429e1dc64000df8 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 +help_all 0 10c38670feb133f4723b29fdfb60d198aa09fb2ecc8200d8c41d92f371ce939b e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 help_one 0 938ff4f8802a8f5d9cb10f9a1f3804ac755be44c8ab00e24e45bfe8255224a21 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855