From 475fa6d87a2d4b8712ab764fac90d83da63151c3 Mon Sep 17 00:00:00 2001 From: Mark Ridley <210189+maudlin@users.noreply.github.com> Date: Wed, 24 Jun 2026 00:40:02 +0100 Subject: [PATCH] =?UTF-8?q?feat(scc):=20the=20honest=20banner=20=E2=80=94?= =?UTF-8?q?=20coverage-by-category=20+=20foreign-language=20concentration?= =?UTF-8?q?=20caveat=20(#117)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plan 0002 Phase 3. Makes first-party exclusions visible and actionable, and catches the markerless flat-vendored case the auto-exclude deliberately can't. - detection.json coverage-by-category (schema 1.5): tracked / firstParty / pctExcluded, and excluded.{generated,authorDeclared,convention} over the all-extension keep-set the scc engines use — surfaced on the console and the report headline so a shrunk scan never reads as "nothing here". Wires in the previously-unsurfaced exclusion counts via staged deltas in build_scc_keepset. - single-directory concentration caveat (plan §6.5): a dir dominated by a language FOREIGN to the repo's primary is flagged as vendored-looking, with the exact one-line fix. Language-aware on purpose — pure share flags the primary source dir on every normal repo (a false positive found in end-to-end testing); the foreign-language test matches the spike evidence (dotCMS webapp/html/js is JS while the codebase is Java) and stays silent on monolithic-source repos. Pure transform in lib/scc-concentration.jq; threshold CHECKUP_CONCENTRATION_PCT (default 25); advisory only, never auto-excluded. - generated-marker exclusion flipped DEFAULT-ON with a loud banner + the enumerated raw/…generated list; kill-switch CHECKUP_EXCLUDE_GENERATED=0. The shrink is announced, never silent. Tests: language-aware concentration (incl. the primary-dir-not-flagged guard) and the coverage-by-category count partition, both env-independent / determinism-safe (#96); flipped the Phase 2 default expectations. Docs: README env vars, architecture.md schema 1.5, CHANGELOG. Refs #107. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 24 ++++++++++--- README.md | 4 ++- bin/checkup-report.sh | 2 ++ bin/checkup.sh | 50 +++++++++++++++++++++++++-- docs/architecture.md | 15 ++++++-- lib/scc-concentration.jq | 65 +++++++++++++++++++++++++++++++++++ lib/scc-inventory.sh | 14 ++++++++ lib/source-inventory.sh | 46 ++++++++++++++++++------- test/scc-inventory.test.sh | 50 +++++++++++++++++++++++++++ test/source-inventory.test.sh | 55 ++++++++++++++++++++++------- 10 files changed, 291 insertions(+), 34 deletions(-) create mode 100644 lib/scc-concentration.jq diff --git a/CHANGELOG.md b/CHANGELOG.md index f02fc6d..c8e669e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,8 +13,24 @@ across the whole thing, honestly*. ### Added -- **Generated-marker exclusion (opt-in)** (#114, plan 0002 Phase 2): with - `CHECKUP_EXCLUDE_GENERATED=1`, files whose head carries a **banner-shaped** +- **The honest banner — first-party coverage, made visible** (#117, plan 0002 + Phase 3): `detection.json` now reports first-party coverage **by category** + (`coverage.tracked` / `firstParty` / `pctExcluded`, and `excluded.{generated, + authorDeclared,convention}`), surfaced on the console and the report headline, so + a shrunk scan can never read as "nothing here" — **"% excluded" is itself a + signal**. A **single-directory concentration** caveat catches the *markerless* + flat-vendored case no marker announces (the dotCMS class): a directory dominated + by a language **foreign** to the repo's primary is flagged as vendored-looking — + language-aware, so the primary source dir is never mistaken for vendored — and the + banner prints the exact one-line fix (`CHECKUP_EXCLUDE='/*'`). Threshold via + `CHECKUP_CONCENTRATION_PCT` (default 25); advisory only, never auto-excluded. + `detection.json` schema → **1.5**. +- **Generated-marker exclusion is now default-on** (#117): the banner-shaped marker + exclusion (#114, below) ships **on by default** — announced by a loud banner + naming the count plus the enumerated `raw/…​.generated` list — with the kill-switch + `CHECKUP_EXCLUDE_GENERATED=0` to restore the whole tree. The shrink is never silent. +- **Generated-marker exclusion** (#114, plan 0002 Phase 2): files whose head + carries a **banner-shaped** generated marker (Go `// Code generated … DO NOT EDIT.`, a comment-leader `@generated`, or C# ``) are dropped from **both** the lizard inventory and the scc keep-set — the lever that stops a codegen-heavy repo from @@ -22,8 +38,8 @@ across the whole thing, honestly*. #105 lizard blow-ups. Markers are case-sensitive and anchored (not the loose substrings a draft used), so first-party code that merely *quotes* a marker — or owns a bare `DO NOT EDIT` (IaC, hand-written migrations) — is kept. Every drop is - **enumerated** to `raw/…​.generated` (loud, not a silent count). Default off until - field-tested; surfacing "% generated" as a headline signal is Phase 3. + **enumerated** to `raw/…​.generated` (loud, not a silent count). Default-on since + Phase 3 (#117). - **First-party source — the scc-based engines honour the inventory** (#109, plan 0002 Phase 1; closes #18): `codebase-stats`, stack-identity, the scc complexity arm and tech-viability now filter + re-aggregate a single diff --git a/README.md b/README.md index f63cedd..3fc3a9a 100644 --- a/README.md +++ b/README.md @@ -294,7 +294,9 @@ finding shape and the rest of the substrate carries it through unchanged. | `CHECKUP_MODE` | closing verdict (`checkup.sh` + renderer) | `tailored` | `tailored` (a repo you own & tune): verdict framed for your own codebase ("where to focus next"); a low score exits non-zero as a quality signal you may act on — not a deploy gate. `audit` (a repo you don't own / due diligence): informational only, framed as "where to invest", **always exits 0**. checkup never gates ([ADR-0009](docs/decisions/0009-deterministic-health-localiser.md)). | | `CHECKUP_SRC_ROOTS` | complexity + git-axis sections | whole tree (VCS-tracked source) | NARROWS the complexity + git-forensics scan to specific space-separated roots (e.g. `app cmd`). By default checkup assesses **all** VCS-tracked source ([honest coverage](docs/architecture.md)); set this only to focus the scan or speed up a very large monorepo. | | `CHECKUP_FORENSIC_SINCE` | git-axis sections | `6.months.ago` | `git log --since` window for hotspots / change-coupling / bug-fix-density. Widen (e.g. `2.years.ago`) for repos with sparse recent history; an empty window degrades to `skip`, never a false `pass`. | -| `CHECKUP_EXCLUDE` | lizard complexity + duplication scans | unset | Extra space-separated fnmatch globs excluded from the lizard scans, on top of the built-in generated/vendored defaults (node_modules, migrations, snapshots, `*.min.*`, …). | +| `CHECKUP_EXCLUDE` | source inventory (all scanners + scc/identity/stats) | unset | Extra space-separated fnmatch globs excluded from the whole inventory — complexity, duplication **and** the scc-based stats/identity (#109) — on top of the built-in generated/vendored defaults (node_modules, migrations, snapshots, `*.min.*`, …). Also settable as a top-level `exclude:` list in `.checkup.yml`. | +| `CHECKUP_EXCLUDE_GENERATED` | source inventory (generated-marker pass) | on (set `0` to disable) | Drop files carrying a banner-shaped generated marker (Go `// Code generated … DO NOT EDIT.`, comment-leader `@generated`, C# ``). **Default-on** since v0.2.0; the dropped set is enumerated to `raw/…generated` and announced by a loud banner. `CHECKUP_EXCLUDE_GENERATED=0` keeps the whole tree. | +| `CHECKUP_CONCENTRATION_PCT` | coverage banner (single-dir concentration) | `25` | Threshold (% of all first-party code) at which one directory is flagged as a possible vendored tree — the banner names the directory and prints the exact `CHECKUP_EXCLUDE` snippet. Advisory only (never auto-excluded). | | `CHECKUP_SHELL_DIRS` | `shellcheck` section | `scripts .husky .githooks .claude/hooks` | Space-separated dirs to search for shell scripts. Missing dirs are skipped silently. | | `HADOLINT_DOCKERFILE` | `hadolint` section | auto-detect `Dockerfile*` at root | Override the Dockerfile filename when it is named non-conventionally. | | `MUTATION_TEST` | `mutation` section | unset (skipped) | Set to `1` to enable Stryker; opt-in because mutation testing is slow (~2 min). | diff --git a/bin/checkup-report.sh b/bin/checkup-report.sh index 000c31e..04c8562 100755 --- a/bin/checkup-report.sh +++ b/bin/checkup-report.sh @@ -98,6 +98,8 @@ if [ -f "$OUT_DIR/detection.json" ]; then COVERAGE_TXT=$(jq -r ' .coverage // empty | "**Coverage:** \(.assessedFiles) source files assessed · scope: \(.scope) · excludes via \(.exclusionSource)" + + (if (.excluded.total // 0) > 0 then " · \(.firstParty)/\(.tracked) first-party (\(.pctExcluded)% excluded: \(.excluded.generated) generated, \(.excluded.authorDeclared) author-declared, \(.excluded.convention) convention)" else "" end) + + (if .concentration then " · ⚠️ \(.concentration.dir)/ is \(.concentration.pct)% of code (\(.concentration.files) files) \(.concentration.lang) tree vs \(.concentration.repoLang) codebase — exclude if vendored: CHECKUP_EXCLUDE=\u0027\(.concentration.dir)/*\u0027" else "" end) + (if .narrowed then " · ⚠️ scope NARROWED by CHECKUP_SRC_ROOTS" else "" end) + (if (.unmeasured // []) | length > 0 then " · ⚠️ not measured: " + ((.unmeasured) | join("; ")) else "" end) ' "$OUT_DIR/detection.json" 2>/dev/null) diff --git a/bin/checkup.sh b/bin/checkup.sh index 1521ad4..278984d 100755 --- a/bin/checkup.sh +++ b/bin/checkup.sh @@ -456,6 +456,29 @@ COVERAGE_NARROWED=false; [ -n "${CHECKUP_SRC_ROOTS:-}" ] && COVERAGE_NARROWED=tr # can't run, #79) — surfaced so the gap is loud, never a silent false-pass. COVERAGE_UNMEASURED=$(printf '%s\n' "${CPLX_UNMEASURED[@]}" | jq -R . | jq -s 'map(select(length>0))') +# Coverage-by-category (#117 Phase 3): attribute every excluded file to a source +# (convention / author-declared / generated markers) over the ALL-extension keep-set +# — the same denominator scc/identity use — and surface "% excluded" as a first-class +# signal. Built by build_scc_keepset; absent on a non-VCS / no-scc target → 0s. +COVERAGE_TRACKED=${SCC_CANDIDATE_COUNT:-0} +COVERAGE_FIRSTPARTY=${SCC_KEEP_COUNT:-0} +COVERAGE_EXC_GENERATED=${GENERATED_EXCLUDED_COUNT:-0} +COVERAGE_EXC_AUTHOR=${GITATTR_EXCLUDED_COUNT:-0} +COVERAGE_EXC_CONVENTION=${CONVENTION_EXCLUDED_COUNT:-0} +COVERAGE_EXC_TOTAL=$(( COVERAGE_EXC_GENERATED + COVERAGE_EXC_AUTHOR + COVERAGE_EXC_CONVENTION )) +COVERAGE_PCT_EXCLUDED=0 +[ "$COVERAGE_TRACKED" -gt 0 ] && COVERAGE_PCT_EXCLUDED=$(( COVERAGE_EXC_TOTAL * 100 / COVERAGE_TRACKED )) + +# Single-directory concentration (#117 Phase 3, plan §6.5): the markerless flat- +# vendored detector. Re-uses the one scc --by-file walk (ensured during detection, +# above) filtered to the keep-set; null when scc is absent or no dir dominates. +COVERAGE_CONCENTRATION=null +if [ "${SCC_BYFILE_OK:-false}" = true ]; then + COVERAGE_CONCENTRATION=$(scc_concentration "$SCC_KEEP_JSON" "${CHECKUP_CONCENTRATION_PCT:-25}" < "$SCC_BYFILE" 2>/dev/null || echo null) + [ -z "$COVERAGE_CONCENTRATION" ] && COVERAGE_CONCENTRATION=null + echo "$COVERAGE_CONCENTRATION" > "$RAW_DIR/concentration.json" +fi + # Topology (#78): the scan root is a hypothesis. Tell a single package from a # declared workspace (healthy) from an UNDECLARED fan-out (a thin orchestrator # root over real packages one level down — a mild structural smell, and the case @@ -501,17 +524,24 @@ jq -n \ --arg scope "${SOURCE_SCOPE:-unknown}" --arg excl "$COVERAGE_EXCL" \ --argjson byArea "${COVERAGE_BY_AREA:-{\}}" --argjson narrowed "$COVERAGE_NARROWED" \ --argjson unmeasured "${COVERAGE_UNMEASURED:-[]}" \ + --argjson tracked "$COVERAGE_TRACKED" --argjson firstParty "$COVERAGE_FIRSTPARTY" \ + --argjson excGen "$COVERAGE_EXC_GENERATED" --argjson excAuthor "$COVERAGE_EXC_AUTHOR" \ + --argjson excConv "$COVERAGE_EXC_CONVENTION" --argjson excTotal "$COVERAGE_EXC_TOTAL" \ + --argjson pctExcluded "$COVERAGE_PCT_EXCLUDED" --argjson concentration "$COVERAGE_CONCENTRATION" \ --arg toposhape "$TOPO_SHAPE" --argjson topowstool "$TOPO_WSTOOL_JSON" \ --arg topolock "$TOPO_ROOT_LOCK" --arg toporeal "$TOPO_ROOT_REAL" \ --argjson toporoots "$TOPO_ROOTS_JSON" --argjson topochildren "$TOPO_CHILD_COUNT" \ --argjson topocapped "$TOPO_CAPPED" ' - {schemaVersion:"1.4", + {schemaVersion:"1.5", primary: (if $primary=="" then null else $primary end), primaryConfidence: $conf, sccBreakdownAvailable: ($sccok=="true"), stacks: $stacks, manifests: $manifests, engines: {complexity:{engine:$ec, reason:$cr, slices:$slices}, duplication:{engine:$ed, reason:$dr}}, - coverage: {assessedFiles:$assessed, scope:$scope, exclusionSource:$excl, narrowed:$narrowed, byArea:$byArea, unmeasured:$unmeasured}, + coverage: {assessedFiles:$assessed, scope:$scope, exclusionSource:$excl, narrowed:$narrowed, byArea:$byArea, unmeasured:$unmeasured, + tracked:$tracked, firstParty:$firstParty, pctExcluded:$pctExcluded, + excluded:{generated:$excGen, authorDeclared:$excAuthor, convention:$excConv, total:$excTotal}, + concentration:$concentration}, topology: {shape:$toposhape, workspaceTool:$topowstool, rootHasLockfile:($topolock=="true"), rootHasRealScripts:($toporeal=="true"), assessmentRoots:$toporoots, childCount:$topochildren, capped:$topocapped}, overridden: ($overridden=="true")}' > "$OUT_DIR/detection.json" @@ -526,6 +556,22 @@ echo -e " Cross-stack checks always run (secrets, SAST, forensics, stats, docs COVERAGE_NOTE=" 📐 Coverage: ${SOURCE_FILE_COUNT:-0} source files assessed (scope: ${SOURCE_SCOPE:-unknown}, excludes via ${COVERAGE_EXCL})" [ "$COVERAGE_NARROWED" = true ] && COVERAGE_NOTE="$COVERAGE_NOTE — NARROWED by CHECKUP_SRC_ROOTS" echo -e "$COVERAGE_NOTE" +# First-party split (#117 Phase 3): name what was excluded, by category, so the +# shrink is never silent. "% excluded" is itself a signal (a codegen-heavy repo). +if [ "$COVERAGE_EXC_TOTAL" -gt 0 ]; then + echo -e " 🧮 First-party: ${COVERAGE_FIRSTPARTY}/${COVERAGE_TRACKED} files (${COVERAGE_PCT_EXCLUDED}% excluded — ${COVERAGE_EXC_GENERATED} generated, ${COVERAGE_EXC_AUTHOR} author-declared, ${COVERAGE_EXC_CONVENTION} convention)" +fi +# Loud generated banner (#117 §10): default-on exclusion must announce itself + the +# enumerated list + the kill-switch, so a reader never mistakes the shrink for the truth. +if [ "${COVERAGE_EXC_GENERATED:-0}" -gt 0 ]; then + echo -e " ${YELLOW}🤖 Excluded ${COVERAGE_EXC_GENERATED} generated file(s)${NC} (banner-shaped markers) — enumerated in raw/scc-keep.lst.generated · keep them with CHECKUP_EXCLUDE_GENERATED=0" +fi +# Single-directory concentration caveat (#117 §6.5): a dir dominated by a language +# foreign to the repo's primary — looks vendored. Name it + the one-line fix. +if [ "$COVERAGE_CONCENTRATION" != "null" ]; then + read -r CONC_DIR CONC_PCT CONC_FILES CONC_LANG CONC_REPOLANG < <(echo "$COVERAGE_CONCENTRATION" | jq -r '"\(.dir) \(.pct) \(.files) \(.lang) \(.repoLang)"') + echo -e " ${YELLOW}📦 Concentration:${NC} ${CONC_DIR}/ is ${CONC_PCT}% of all code (${CONC_FILES} files, mostly ${CONC_LANG}; the codebase is mostly ${CONC_REPOLANG}) — looks vendored. To exclude: CHECKUP_EXCLUDE='${CONC_DIR}/*' (or add to .checkup.yml exclude:)" +fi [ "${#CPLX_UNMEASURED[@]}" -gt 0 ] && echo -e " ${YELLOW}⚠️ Not measured:${NC} ${CPLX_UNMEASURED[*]}" case "$TOPO_SHAPE" in undeclared-fan-out) echo -e " ${YELLOW}🧩 Topology:${NC} undeclared fan-out — ${TOPO_CHILD_COUNT} sub-package(s) below an orchestrator root (${TOPO_ASSESSMENT_ROOTS[*]}); the root scan can't see them" ;; diff --git a/docs/architecture.md b/docs/architecture.md index 82f8c6f..95c510b 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -131,7 +131,7 @@ The plan is printed for a human and persisted to `detection.json` (in `OUT_DIR`, ```jsonc { - "schemaVersion": "1.4", + "schemaVersion": "1.5", "primary": "node", // largest stack, or null when ambiguous "primaryConfidence": "high", // high (manifest + dominant) | medium | low "sccBreakdownAvailable": true, // false → degraded to manifest/presence signal @@ -152,10 +152,21 @@ The plan is printed for a human and persisted to `detection.json` (in `OUT_DIR`, // dir; narrowed = true when CHECKUP_SRC_ROOTS restricted the scope; unmeasured // names what a routed engine could NOT cover (e.g. JS/TS complexity when no // resolvable ESLint config — #79), so the gap is explicit, not a false pass. + // tracked/firstParty/excluded/pctExcluded (#117, schema 1.5) attribute every + // excluded file to a category (generated markers | author-declared .gitattributes + // | convention globs) over the ALL-extension keep-set scc/identity use — "% + // excluded" is itself a signal. concentration (plan §6.5) names a single + // directory that dominates the first-party code (a possible markerless vendored + // tree) + the one-line fix; language-aware (a dir dominated by a language foreign + // to the repo's primary), so the source dir itself is never mistaken for vendored; + // null when scc is absent or no foreign dir dominates. "coverage": { "assessedFiles": 412, "scope": "git", "exclusionSource": ".gitignore", "narrowed": false, "byArea": { "src": 280, "server": 110, "scripts": 22 }, - "unmeasured": [] // e.g. ["JS/TS complexity (no resolvable root ESLint config)"] + "unmeasured": [], // e.g. ["JS/TS complexity (no resolvable root ESLint config)"] + "tracked": 4820, "firstParty": 470, "pctExcluded": 90, + "excluded": { "generated": 4100, "authorDeclared": 180, "convention": 70, "total": 4350 }, + "concentration": null // or { "dir": "webapp/html/js", "code", "files", "totalCode", "pct": 25, "lang": "JavaScript", "repoLang": "Java" } }, // Package topology (#78): the scan root is a hypothesis. shape distinguishes a // single package from a declared workspace (healthy) from an UNDECLARED fan-out diff --git a/lib/scc-concentration.jq b/lib/scc-concentration.jq new file mode 100644 index 0000000..6afe62b --- /dev/null +++ b/lib/scc-concentration.jq @@ -0,0 +1,65 @@ +# scc-concentration.jq — single-directory concentration caveat (plan 0002 §6.5, +# #117 Phase 3). The detector for MARKERLESS flat-vendored code that no generated +# marker announces and no convention dir catches (the dotCMS class: a committed +# JS library tree sitting in the first-party keep-set, skewing identity). Too fuzzy +# to auto-exclude — surfaced as a BANNER that names the directory and prints the +# one-line fix, never a silent drop. +# +# LANGUAGE-AWARE, not raw share: the spike (§9) showed the tell is a dir dominated +# by a language FOREIGN to the repo's primary (dotCMS: webapp/html/js is JavaScript +# while the codebase is Java). Pure share alone flags the primary SOURCE directory +# on any normal repo (it is the largest dir) — a false positive that would fire +# everywhere. So a directory is a candidate only when its dominant language differs +# from the repo's dominant language: that is what "vendored-looking" means here. +# +# Input: scc --by-file --format json (array of language objects, each .Files[]). +# Args: --slurpfile keep (JSON array of TARGET-relative keep paths, +# ALL extensions — same set as scc-aggregate) +# --argjson pct (threshold percent of total code, e.g. 25) +# Output: the most-SPECIFIC (depth-maximal) directory whose share of the kept code +# is ≥ pct AND whose dominant language ≠ the repo's, as +# { dir, code, files, totalCode, pct, lang, repoLang }, or null. +# +# Depth-maximal, not share-maximal: a vendored tree's own subdirs each fall below +# the threshold while the tree's own root is the deepest prefix still above it — so +# "deepest ≥ threshold" names the vendored ROOT (e.g. .../webapp/html/js), the most +# precise exclude path, rather than a shallow ancestor that also sweeps in siblings. +# Deterministic (#96): sort by [depth, share, dir], take the last. + +( ($keep[0] // []) | map({ key: sub("^\\./"; ""), value: true }) | from_entries ) as $k +| [ .[].Files[]? + | { loc: (.Location | sub("^\\./"; "")), code: (.Code // 0), lang: (.Language // "") } + | select( $k[.loc] // false ) ] as $files +| ( $files | map(.code) | add // 0 ) as $total +| if $total <= 0 then null + else + # The repo's dominant language (by kept code) — the baseline a candidate dir + # must differ from to read as "vendored-looking". + ( $files | group_by(.lang) + | map({ lang: .[0].lang, code: (map(.code) | add) }) + | sort_by([ -.code, .lang ]) | .[0].lang ) as $repoLang + | ( $files + | map( . as $f + | ($f.loc | split("/")) as $parts + | ($parts[0:-1]) as $dirs # drop the filename + | [ range(1; ($dirs|length)+1) as $n + | { dir: ($dirs[0:$n] | join("/")), code: $f.code, lang: $f.lang } ] ) + | add // [] + | group_by(.dir) + | map( { dir: .[0].dir, + code: (map(.code) | add), + files: length, + share: ((map(.code) | add) / $total), + # this dir's own dominant language + lang: ( group_by(.lang) + | map({ lang: .[0].lang, code: (map(.code) | add) }) + | sort_by([ -.code, .lang ]) | .[0].lang ) } ) + | map( select( .share >= ($pct / 100) and .files >= 2 and .lang != $repoLang ) ) + | sort_by( [ (.dir | split("/") | length), .share, .dir ] ) + | last ) as $top + | if $top == null then null + else { dir: $top.dir, code: $top.code, files: $top.files, + totalCode: $total, pct: (($top.share * 100) | floor), + lang: $top.lang, repoLang: $repoLang } + end + end diff --git a/lib/scc-inventory.sh b/lib/scc-inventory.sh index 7ba9cc2..9396a79 100644 --- a/lib/scc-inventory.sh +++ b/lib/scc-inventory.sh @@ -77,6 +77,20 @@ scc_breakdown_toplangs() { .[0:$n] | map("\(.Name) \(.Code)") | join(", ")' } +# scc_concentration [pct] +# stdin: scc --by-file --format json +# stdout: the single-directory concentration finding (plan 0002 §6.5, #117) as +# { dir, code, files, totalCode, pct, lang, repoLang }, or "null" when no +# kept directory holds ≥ pct (default 25) of the kept code IN A LANGUAGE +# FOREIGN to the repo's primary. Language-aware so the primary source dir +# (the largest dir on any normal repo) is not mistaken for a vendored tree +# — the markerless-flat-vendored detector (the dotCMS class). Advisory +# banner input. Pure transform in lib/scc-concentration.jq (unit-tested). +scc_concentration() { + local keepfile="$1" pct="${2:-25}" + jq --slurpfile keep "$keepfile" --argjson pct "$pct" -f "$CHECKUP_HOME/lib/scc-concentration.jq" +} + # scc_perfile_findings # stdin: scc --by-file --format json # stdout: complexity findings array (file/line/ccn/lines/code/severity/message), diff --git a/lib/source-inventory.sh b/lib/source-inventory.sh index 53871c3..f2b9713 100644 --- a/lib/source-inventory.sh +++ b/lib/source-inventory.sh @@ -168,6 +168,7 @@ _apply_gitattr_excludes() { while IFS= read -r -d '' p && IFS= read -r -d '' a && IFS= read -r -d '' v; do case "$v" in set|true) _flagged["$p"]=1 ;; esac done < <(git check-attr -z --stdin linguist-generated linguist-vendored < "$lst" 2>/dev/null) + GITATTR_EXCLUDED_COUNT="${#_flagged[@]}" # author-declared (coverage signal, #117) [ "${#_flagged[@]}" -eq 0 ] && return 0 local out="$lst.attr" while IFS= read -r -d '' p; do @@ -178,15 +179,18 @@ _apply_gitattr_excludes() { } # Drop files whose head carries a BANNER-SHAPED generated marker (#114, plan 0002 -# Phase 2) — the corvus lever. OPT-IN via CHECKUP_EXCLUDE_GENERATED (default off) -# so the regex is field-tested with zero default-behaviour change before any flip -# to default-on. Rewrites the NUL list at $1 in place (order-preserving) and -# ENUMERATES the dropped files to ".generated" (loud, not a silent count — -# the guard that earns the subtractive design; over-exclusion is the danger). -# Sets GENERATED_EXCLUDED_COUNT. Runs with cwd == TARGET so the relative paths open. +# Phase 2) — the corvus lever. DEFAULT-ON since Phase 3 (#117): the markers are +# field-proven and the shrink is announced by a loud banner, so excluding generated +# code is the honest default. KILL-SWITCH: `CHECKUP_EXCLUDE_GENERATED=0` (also +# false/no/off) restores the whole tree. Rewrites the NUL list at $1 in place +# (order-preserving) and ENUMERATES the dropped files to ".generated" (loud, +# not a silent count — the guard that earns the subtractive design; over-exclusion +# is the danger). Sets GENERATED_EXCLUDED_COUNT. Runs with cwd == TARGET so the +# relative paths open. _apply_generated_excludes() { local lst="$1" - [ -n "${CHECKUP_EXCLUDE_GENERATED:-}" ] && [ -s "$lst" ] || return 0 + case "${CHECKUP_EXCLUDE_GENERATED:-1}" in 0|false|no|off) return 0 ;; esac + [ -s "$lst" ] || return 0 local flagged="$lst.generated" # grep -l short-circuits on the first match per file (the marker is at the head), # so even a mostly-generated tree is cheap; -I skips binaries; LC_ALL=C makes the @@ -209,20 +213,38 @@ _apply_generated_excludes() { # engines filter their --by-file output against this (lib/scc-inventory.sh), so # stats/identity reflect first-party code, not the whole tree (#109). Same tiers # as build_source_inventory; run AFTER resolve_scan_roots, cwd == TARGET. +# +# Subtraction is STAGED so the coverage signal (#117 Phase 3) can attribute every +# excluded file to a category — convention (builtin globs + CHECKUP_EXCLUDE), +# author-declared (.gitattributes), generated (markers) — computed from the count +# delta at each stage (robust to the helpers' internals). Sets, over the ALL- +# extension candidate set: SCC_CANDIDATE_COUNT (everything tracked), SCC_KEEP_COUNT +# (kept), CONVENTION_EXCLUDED_COUNT, GITATTR_EXCLUDED_COUNT, GENERATED_EXCLUDED_COUNT. build_scc_keepset() { - local lst="$RAW_DIR/scc-keep.lst" + local lst="$RAW_DIR/scc-keep.lst" raw="$RAW_DIR/scc-keep.raw" SCC_KEEP_JSON="$RAW_DIR/scc-keep.json" mkdir -p "$RAW_DIR" + # Enumerate ALL tracked candidates first (every extension), so each subtraction + # stage below has an honest denominator. if [ "${GIT_OK:-false}" = true ]; then - git ls-files -z -- "${SCAN_ROOTS[@]}" 2>/dev/null | _filter_keep > "$lst" + git ls-files -z -- "${SCAN_ROOTS[@]}" 2>/dev/null > "$raw" elif command -v fd > /dev/null 2>&1; then - fd --type f --hidden --no-follow --print0 . "${SCAN_ROOTS[@]}" 2>/dev/null | _filter_keep > "$lst" + fd --type f --hidden --no-follow --print0 . "${SCAN_ROOTS[@]}" 2>/dev/null > "$raw" else find "${SCAN_ROOTS[@]}" \( -name node_modules -o -name .git \) -prune -o \ - -type f -print0 2>/dev/null | _filter_keep > "$lst" + -type f -print0 2>/dev/null > "$raw" fi + SCC_CANDIDATE_COUNT=$(tr -cd '\0' < "$raw" | wc -c | tr -d ' ') + _filter_keep < "$raw" > "$lst" # convention globs + CHECKUP_EXCLUDE + rm -f "$raw" + local after_conv; after_conv=$(tr -cd '\0' < "$lst" | wc -c | tr -d ' ') + CONVENTION_EXCLUDED_COUNT=$(( SCC_CANDIDATE_COUNT - after_conv )) _apply_gitattr_excludes "$lst" # author-declared generated/vendored (.gitattributes) - _apply_generated_excludes "$lst" # generated-file markers (#114, opt-in CHECKUP_EXCLUDE_GENERATED) + local after_attr; after_attr=$(tr -cd '\0' < "$lst" | wc -c | tr -d ' ') + GITATTR_EXCLUDED_COUNT=$(( after_conv - after_attr )) + _apply_generated_excludes "$lst" # generated-file markers (#114, default-on; CHECKUP_EXCLUDE_GENERATED=0 disables) + SCC_KEEP_COUNT=$(tr -cd '\0' < "$lst" | wc -c | tr -d ' ') + GENERATED_EXCLUDED_COUNT=$(( after_attr - SCC_KEEP_COUNT )) jq -Rs 'split("\u0000") | map(select(length > 0))' < "$lst" > "$SCC_KEEP_JSON" 2>/dev/null \ || printf '[]' > "$SCC_KEEP_JSON" return 0 diff --git a/test/scc-inventory.test.sh b/test/scc-inventory.test.sh index bf5f561..c09e172 100644 --- a/test/scc-inventory.test.sh +++ b/test/scc-inventory.test.sh @@ -124,6 +124,56 @@ jq '[ .[] | {Name, Files: (.Files|reverse)} ] | reverse' "$BYFILE" > "$TMP/shuff B2=$(scc_breakdown "$KEEP" < "$TMP/shuffled.json") assert_eq "breakdown stable under input reorder" "$(printf '%s' "$BREAKDOWN" | jq -S -c '.')" "$(printf '%s' "$B2" | jq -S -c '.')" +echo "" +echo "single-dir concentration (§6.5, #117): foreign-language dir, depth-maximal, deterministic" +# The repo is mostly C# (the real code under src/); a markerless flat-vendored JS +# tree sits under webapp/html/js (still in the keep-set — no marker/convention +# catches it). The detector must name the FOREIGN-language tree, not the larger +# primary C# dir (the false positive that would fire on every normal repo). +CONC_BYFILE="$TMP/conc.json" +cat > "$CONC_BYFILE" <<'JSON' +[ + {"Name":"C#","Files":[ + {"Location":"src/A.cs","Language":"C#","Code":500,"Complexity":50,"Lines":600}, + {"Location":"src/B.cs","Language":"C#","Code":500,"Complexity":50,"Lines":600} + ]}, + {"Name":"JavaScript","Files":[ + {"Location":"./webapp/html/js/a.js","Language":"JavaScript","Code":200,"Complexity":5,"Lines":2000}, + {"Location":"webapp/html/js/b.js","Language":"JavaScript","Code":200,"Complexity":5,"Lines":2000} + ]} +] +JSON +CONC_KEEP="$TMP/conc-keep.json" +echo '["src/A.cs","src/B.cs","webapp/html/js/a.js","webapp/html/js/b.js"]' > "$CONC_KEEP" + +CONC=$(scc_concentration "$CONC_KEEP" 25 < "$CONC_BYFILE") +# 400/1400 = 28% under webapp/html/js; JS ≠ C# (repo primary) → flagged. src (71%, +# C# = primary) is NOT flagged despite being larger. +assert_eq "names the FOREIGN-language tree, not the bigger primary dir" "webapp/html/js" "$(printf '%s' "$CONC" | jq -r '.dir')" +assert_eq "share floored to pct" "28" "$(printf '%s' "$CONC" | jq -r '.pct')" +assert_eq "file count under the dir" "2" "$(printf '%s' "$CONC" | jq -r '.files')" +assert_eq "totalCode = kept code" "1400" "$(printf '%s' "$CONC" | jq -r '.totalCode')" +assert_eq "reports the foreign language" "JavaScript" "$(printf '%s' "$CONC" | jq -r '.lang')" +assert_eq "reports the repo's language" "C#" "$(printf '%s' "$CONC" | jq -r '.repoLang')" + +echo "" +echo "concentration: same-language large dir is NOT flagged (no primary-dir false positive)" +# Keep ONLY the C# — now src is 100% C# = repoLang → nothing foreign → null. +echo '["src/A.cs","src/B.cs"]' > "$TMP/conc-cs.json" +assert_eq "all-primary-language repo → null" "null" "$(scc_concentration "$TMP/conc-cs.json" 25 < "$CONC_BYFILE")" + +echo "" +echo "concentration: threshold respected + degrade cases" +assert_eq "pct above share → null" "null" "$(scc_concentration "$CONC_KEEP" 40 < "$CONC_BYFILE")" +assert_eq "empty keep → null" "null" "$(scc_concentration "$TMP/empty.json" 25 < "$CONC_BYFILE")" + +echo "" +echo "concentration: deterministic under input reorder (#96)" +jq '[ .[] | {Name, Files:(.Files|reverse)} ] | reverse' "$CONC_BYFILE" > "$TMP/conc-shuf.json" +assert_eq "stable under reorder" \ + "$(scc_concentration "$CONC_KEEP" 25 < "$CONC_BYFILE" | jq -S -c '.')" \ + "$(scc_concentration "$CONC_KEEP" 25 < "$TMP/conc-shuf.json" | jq -S -c '.')" + echo "" echo "Results: $PASS passed, $FAIL failed" [ "$FAIL" -eq 0 ] diff --git a/test/source-inventory.test.sh b/test/source-inventory.test.sh index 274cf4e..82768eb 100755 --- a/test/source-inventory.test.sh +++ b/test/source-inventory.test.sh @@ -156,7 +156,7 @@ else fi echo "" -echo "generated-marker exclusion: opt-in, banner-shaped, keeps marker-quoting first-party (#114)" +echo "generated-marker exclusion: default-on (#117), banner-shaped, keeps marker-quoting first-party (#114)" if command -v git > /dev/null 2>&1; then G="$HOME/.checkup-gen.$$"; rm -rf "$G"; mkdir -p "$G/gen" "$G/src" ( cd "$G" @@ -168,25 +168,54 @@ if command -v git > /dev/null 2>&1; then printf '// This documents the @generated convention\npackage y\n' > src/notes.go # prose → keep printf '# DO NOT EDIT - managed by our own tool\nx=1\n' > src/migrate.py # bare DO NOT EDIT → keep git add -A && git commit -qm init >/dev/null 2>&1 ) - # Opt-in OFF (default): nothing dropped on markers. - off=$( cd "$G"; GIT_OK=true; RAW_DIR="$G/raw"; SCAN_ROOTS=(.); build_source_inventory; tr '\0' '\n' < "$SOURCE_LST" | sort | paste -sd',' - ) - assert_eq "default (opt-in off) keeps everything" \ - "gen/api.pb.go,src/Model.cs,src/app.js,src/migrate.py,src/notes.go,src/widget.js" "$off" - # Opt-in ON: the three banner-shaped generated files drop; the marker-quoting - # first-party files (prose, bare DO NOT EDIT) are kept (Morlock M1). - on=$( cd "$G"; GIT_OK=true; RAW_DIR="$G/raw"; SCAN_ROOTS=(.); CHECKUP_EXCLUDE_GENERATED=1; build_source_inventory - tr '\0' '\n' < "$SOURCE_LST" | sort | paste -sd',' - ) - assert_eq "opt-in drops banner-shaped generated, keeps marker-quoting first-party" \ + # Default ON (#117 flip): the three banner-shaped generated files drop; the + # marker-quoting first-party files (prose, bare DO NOT EDIT) are kept (Morlock M1). + on=$( cd "$G"; GIT_OK=true; RAW_DIR="$G/raw"; SCAN_ROOTS=(.); build_source_inventory; tr '\0' '\n' < "$SOURCE_LST" | sort | paste -sd',' - ) + assert_eq "default-on drops banner-shaped generated, keeps marker-quoting first-party" \ "src/app.js,src/migrate.py,src/notes.go" "$on" - cnt=$( cd "$G"; GIT_OK=true; RAW_DIR="$G/raw"; SCAN_ROOTS=(.); CHECKUP_EXCLUDE_GENERATED=1; build_source_inventory; printf '%s' "$GENERATED_EXCLUDED_COUNT" ) - assert_eq "GENERATED_EXCLUDED_COUNT set" "3" "$cnt" - enum=$( cd "$G"; GIT_OK=true; RAW_DIR="$G/raw"; SCAN_ROOTS=(.); CHECKUP_EXCLUDE_GENERATED=1; build_source_inventory; tr '\0' '\n' < "$RAW_DIR/source-files.lst.generated" | sort | paste -sd',' - ) + # Kill-switch: CHECKUP_EXCLUDE_GENERATED=0 restores the whole tree. + off=$( cd "$G"; GIT_OK=true; RAW_DIR="$G/raw"; SCAN_ROOTS=(.); CHECKUP_EXCLUDE_GENERATED=0; build_source_inventory; tr '\0' '\n' < "$SOURCE_LST" | sort | paste -sd',' - ) + assert_eq "kill-switch (=0) keeps everything" \ + "gen/api.pb.go,src/Model.cs,src/app.js,src/migrate.py,src/notes.go,src/widget.js" "$off" + cnt=$( cd "$G"; GIT_OK=true; RAW_DIR="$G/raw"; SCAN_ROOTS=(.); build_source_inventory; printf '%s' "$GENERATED_EXCLUDED_COUNT" ) + assert_eq "GENERATED_EXCLUDED_COUNT set (default-on)" "3" "$cnt" + enum=$( cd "$G"; GIT_OK=true; RAW_DIR="$G/raw"; SCAN_ROOTS=(.); build_source_inventory; tr '\0' '\n' < "$RAW_DIR/source-files.lst.generated" | sort | paste -sd',' - ) assert_eq "drops enumerated (loud, not silent)" "gen/api.pb.go,src/Model.cs,src/widget.js" "$enum" rm -rf "$G" else echo " ⊘ skipped — git not installed" fi +echo "" +echo "build_scc_keepset: coverage-by-category counts (convention/author/generated, #117)" +if command -v git > /dev/null 2>&1 && command -v jq > /dev/null 2>&1; then + G="$HOME/.checkup-cov.$$"; rm -rf "$G"; mkdir -p "$G/src" "$G/external" "$G/gen" + ( cd "$G" + git init -q && git config user.email t@t && git config user.name t + # NB: a non-builtin dir name (external/), so the author-declared exclusion is + # what catches it — `vendor/` would be caught by the convention stage first. + printf 'external/** linguist-vendored\n' > .gitattributes + printf 'export const a=1;\n' > src/a.ts # first-party (keep) + printf '{"x":1}\n' > src/data.json # first-party config (keep, all-ext) + printf 'export const v=1;\n' > external/lib.js # author-declared vendored → excluded + printf 'export const m=1;\n' > src/x.min.js # convention suffix → excluded + printf '// @generated\nexport const g=1;\n' > gen/api.js # generated marker → excluded + git add -A && git commit -qm init >/dev/null 2>&1 ) + run() { ( cd "$G"; GIT_OK=true; RAW_DIR="$G/raw"; SCAN_ROOTS=(.); build_scc_keepset; printf '%s' "${!1}" ); } + assert_eq "SCC_CANDIDATE_COUNT = 6 (all tracked, incl .gitattributes)" "6" "$(run SCC_CANDIDATE_COUNT)" + assert_eq "CONVENTION_EXCLUDED_COUNT = 1 (.min.js)" "1" "$(run CONVENTION_EXCLUDED_COUNT)" + assert_eq "GITATTR_EXCLUDED_COUNT = 1 (vendor/lib.js)" "1" "$(run GITATTR_EXCLUDED_COUNT)" + assert_eq "GENERATED_EXCLUDED_COUNT = 1 (gen/api.js)" "1" "$(run GENERATED_EXCLUDED_COUNT)" + assert_eq "SCC_KEEP_COUNT = 3 (a.ts, data.json, .gitattributes)" "3" "$(run SCC_KEEP_COUNT)" + # The deltas must partition the candidate set exactly (no double count, no leak). + part=$( cd "$G"; GIT_OK=true; RAW_DIR="$G/raw"; SCAN_ROOTS=(.); build_scc_keepset + echo $(( SCC_KEEP_COUNT + CONVENTION_EXCLUDED_COUNT + GITATTR_EXCLUDED_COUNT + GENERATED_EXCLUDED_COUNT )) ) + assert_eq "kept + excluded categories == candidates" "6" "$part" + rm -rf "$G" +else + echo " ⊘ skipped — git/jq not installed" +fi + echo "" echo "coverage helpers: by-area grouping and exclusion-source label" SOURCE_LST="$TMP/cov.lst"