diff --git a/.agents/skills/euthyna-en/references/fact-producers.md b/.agents/skills/euthyna-en/references/fact-producers.md index 0de4645..041a2ae 100644 --- a/.agents/skills/euthyna-en/references/fact-producers.md +++ b/.agents/skills/euthyna-en/references/fact-producers.md @@ -137,13 +137,15 @@ node /bin/euthyna.js coverage ` `--symbol` is repeatable — query several at once. Add `--file` to restrict to a single file. Go profiles additionally need `--source` (see below). -**Three coverage formats are read** (auto-detected, no parameters needed): +**Five coverage formats are read** (auto-detected, no parameters needed): | Format | Source | Notes | |---|---|---| | c8 / v8-to-istanbul `coverage-final.json` | JS projects (c8 lands at `coverage/coverage-final.json` by default) | Per-file `fnMap` + `f` (invocation-count array) | +| classic istanbul `coverage-final.json` | jest's default provider, nyc | The same `fnMap` + `f` plus a `hash` field (c8 has no `hash`); both resolve through the same code path | | coverage.py JSON (format 3) | Python projects (produced by `coverage json`) | Top-level `{meta, files}`; in `functions`, "function name → list of executed lines", **no invocation counts**. Functions never called are still in the list with empty `executed_lines` | | Go `-coverprofile` (text) | Go projects (produced by `go test -coverprofile`) | A `mode:` header plus one line per block (`path:startL.startC,endL.endC stmts count`), **no function names**. `--source ` is required: the producer reads top-level `func` declarations in the source to map line ranges back to functions | +| lcov tracefile (text) | Rust projects (produced by cargo-llvm-cov / cargo-tarpaulin); gcov / llvm-cov produce the format too | `SF:` opens a record, `end_of_record` closes it; `FN:,` and `FNDA:,` give a function name and its invocation count (**two separate runs, not adjacent pairs**); **the symbol names are mangled**, and the producer anchors the `module::function` path you write against the `` tail segments of Rust v0 mangling, never inside a longer `` pair — a query for `get` does not answer for `cfg_3get` | The Go production commands (Go targets): @@ -180,6 +182,32 @@ uv run --with coverage python -m coverage json -o .json Where the coverage file comes from: if the project produces one, use it. When the project does not, **either run its tests once with the project's own tooling to produce it, or mark the criterion as "not evaluable"**. +The lcov production commands (Rust targets, measured on this machine): + +```powershell +# 0. llvm-cov / llvm-profdata are not on PATH; locate rustc's own copies first +$bin = (rustc --print sysroot) + "\lib\rustlib\x86_64-pc-windows-msvc\bin" +# 1. instrumented build + run the tests +$env:RUSTFLAGS = "-C instrument-coverage" +$env:LLVM_PROFILE_FILE = "\prof\probe-%p-%m.profraw" +cargo test +# 2. merge the profraw files, then export lcov from the test binary +& "$bin\llvm-profdata.exe" merge -sparse \prof\*.profraw -o \probe.profdata +& "$bin\llvm-cov.exe" export --format=lcov --instr-profile=\probe.profdata ` + "-.exe>" --sources src > \lcov.info +# 3. query (lcov needs no --source: the function names are in the FN: lines) +node /bin/euthyna.js coverage ` + --coverage \lcov.info --symbol --file \src\lib.rs +``` + +> lcov-specific caveats (measured on a probe crate): +> 1. **The symbol names are mangled and this producer does not demangle them** (no usable demangler on this machine: `llvm-cov --Xdemangler` was tried with `rust` / `rustfilt` / `llvm-cxxfilt` / `c++filt`, and all four report `could not find the demangler`). Matching relies on the `` tail segments of **Rust v0 mangling** (what `-C instrument-coverage` implies), so write the query as `module::function` (split on `::`, matched per segment). Each segment's name must start at a boundary — the start of the name, right after the mangling's own `_` separator, or at a proven pair boundary — and never strictly inside a longer pair: a query for `get` does not answer for `cfg_3get`. A qualified query is a subsequence test, not a path-exact one. `nested::dup` matches only `6nested3dup`, while a bare `dup` also matches the same-named function in an outer module — each hit is reported as its own fact carrying its own mangled name. An already-mangled query (starting with `_R`/`_Z`) is matched exactly and **never half-decoded**. +> 2. **One file may carry several `FN:` lines pointing at the same line** (generic monomorphisation, closures). That is not a duplicate record. A generic function instantiated twice yields two facts, each carrying its own mangled name. +> 3. When an `FN:` line exists and its `FNDA:` is missing, the result is "not evaluated", **never a default of 0** — inventing a "never invoked" out of thin air is the one direction this producer must not fail in. +> 4. When one `SF:` appears in several records (or several same-named `FNDA:` lines exist), the counts are **summed**; summing can never turn a non-zero into zero, which is what keeps the only actionable conclusion (a count of 0) safe. +> 5. A file absent from the tracefile is reported as **"not evaluated"** (llvm-cov / gcov only emit records for instrumented targets, and `--sources` filters further) — deliberately the opposite of c8's "file absent from the data = never executed". +> 6. The newer `FNL:`/`FNA:` encoding is **explicitly refused** rather than read out of a guessed shape; `SF:` paths are absolute (backslash-separated on Windows). + ### What the output looks like (real capture; one query hitting both result kinds) ``` diff --git a/.agents/skills/euthyna/references/fact-producers.md b/.agents/skills/euthyna/references/fact-producers.md index 242360d..8f72074 100644 --- a/.agents/skills/euthyna/references/fact-producers.md +++ b/.agents/skills/euthyna/references/fact-producers.md @@ -149,13 +149,15 @@ node /bin/euthyna.js coverage ` `--symbol` 可重复，一次查多个。加 `--file` 可限定到某个文件。Go profile 另需 `--source` （见下）。 -**三种覆盖率格式都读**（自动识别，无需参数）： +**五种覆盖率格式都读**（自动识别，无需参数）： | 格式 | 来源 | 说明 | |---|---|---| | c8 / v8-to-istanbul `coverage-final.json` | JS 项目（c8 默认落在 `coverage/coverage-final.json`） | 每文件有 `fnMap` + `f`（调用计数数组） | +| classic istanbul `coverage-final.json` | jest 默认 provider、nyc | 同样的 `fnMap` + `f`，另多一个 `hash` 字段（c8 没有 `hash`），两者走同一条代码路径 | | coverage.py JSON（format 3） | Python 项目（`coverage json` 产出） | 顶层 `{meta, files}`；`functions` 里「函数名 → 已执行行列表」，**无调用计数**。从未调用的函数仍在列表中且 `executed_lines` 为空 | | Go `-coverprofile`（文本） | Go 项目（`go test -coverprofile` 产出） | `mode:` 头 + 每区块一行 `路径:起始行.列,结束行.列 语句数 命中数`，**无函数名**。必须加 `--source <模块根目录>`：产出器读源码里的顶层 `func` 把行区间映射回函数 | +| lcov tracefile（文本） | Rust 项目（cargo-llvm-cov / cargo-tarpaulin 产出），gcov / llvm-cov 也产这个格式 | `SF:` 开记录、`end_of_record` 收尾，`FN:<行>,<符号>` 与 `FNDA:<计数>,<符号>` 给出函数名与调用计数（**两段分开出现，不是相邻配对**）；**符号名是 mangled 的**，产出器按 Rust v0 mangling 的 `<十进制长度><名字>` 尾段锚定匹配你写的 `模块::函数` 路径，且绝不落在更长的 `<长度><名字>` 对内——查 `get` 不会答到 `cfg_3get` | Go 的产出命令（Go 目标）： ```bash @@ -189,6 +191,46 @@ uv run --with coverage python -m coverage json -o <输出路径>.json 覆盖率文件从哪来：项目自己产出就用它。项目不产出覆盖率时，**要么用项目自己的方式跑一次 测试产出它，要么把该判据标为「不可评估」**。 +lcov 的产出命令（Rust 目标，本机实测）： + +```powershell +# 0. llvm-cov / llvm-profdata 不在 PATH 上，先定位 rustc 自带的那一份 +$bin = (rustc --print sysroot) + "\lib\rustlib\x86_64-pc-windows-msvc\bin" +# 1. 插桩构建 + 跑测试 +$env:RUSTFLAGS = "-C instrument-coverage" +$env:LLVM_PROFILE_FILE = "<绝对路径>\prof\probe-%p-%m.profraw" +cargo test +# 2. 合并 profraw，再从测试二进制导出 lcov +& "$bin\llvm-profdata.exe" merge -sparse <绝对路径>\prof\*.profraw -o <绝对路径>\probe.profdata +& "$bin\llvm-cov.exe" export --format=lcov --instr-profile=<绝对路径>\probe.profdata ` + "-.exe>" --sources src > <绝对路径>\lcov.info +# 3. 查询（lcov 不需要 --source：函数名就在 FN: 行里） +node /bin/euthyna.js coverage ` + --coverage <绝对路径>\lcov.info --symbol <模块::函数> --file <绝对路径>\src\lib.rs +``` + +> lcov 专属注意事项（在探针 crate 上实测）： +> 1. **符号名是 mangled 的，本产出器不做 demangle**（本机没有可用的 demangler： +> `llvm-cov --Xdemangler` 试过 `rust` / `rustfilt` / `llvm-cxxfilt` / `c++filt` +> 四个名字全部报 `could not find the demangler`）。匹配靠 **Rust v0 mangling** +> （`-C instrument-coverage` 蕴含的版本）的 `<十进制长度><名字>` 尾段编码， +> 所以查询写 `模块::函数`（`split('::')` 后逐段匹配）。每段的名字必须从段边界起算—— +> 名字开头、mangling 自己的 `_` 分隔符之后、或已证明的 pair 边界——且绝不落在更长的 +> pair 内部：查 `get` 不会答到 `cfg_3get`。带限定的查询是子序列判定而非路径精确判定。 +> `nested::dup` 只命中 `6nested3dup`，裸 `dup` 会同时命中外层模块里的同名函数—— +> 每个命中各自报一条事实并带上自己的 mangled 名。已经 mangled 的查询（`_R`/`_Z` 开头） +> 只做精确匹配，**绝不半解码**。 +> 2. **同一个文件可能有多条 `FN:` 指向同一行**（泛型单态化、闭包），这不是重复记录。 +> 一个泛型函数实例化两次就是两条事实，各自带自己的 mangled 名。 +> 3. `FN:` 行存在而 `FNDA:` 缺失时报「未评估」，**不默认成 0**——凭空造出「从未调用」 +> 是这个产出器最不能犯的方向。 +> 4. 同一 `SF:` 出现多条记录（或同名多条 `FNDA:`）时**计数求和**；求和不会把非零变零， +> 对唯一可行动的结论（计数为 0）安全。 +> 5. 文件不在 tracefile 里报**「未评估」**（llvm-cov / gcov 只为被插桩的目标产记录， +> `--sources` 还会再过滤一层）——这与 c8 的「文件不在数据里 = 从未执行」刻意相反。 +> 6. 认较新的 `FNL:`/`FNA:` 编码时会**明确拒绝**而不是按猜的形状读；`SF:` 路径是绝对 +> 路径（Windows 上是带反斜杠的）。 + ### 输出长什么样（真实捕获，一次查询命中两种结果） ``` diff --git a/AGENTS.md b/AGENTS.md index 39e4106..29c39aa 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -674,8 +674,8 @@ is faithful — that is a reading job, and a mechanical stand-in would only manu | Path | What it is | |---|---| -| `bin/` + `src/` | **Fact producers** (zero-dependency Node CLI). `src/contract.js` is the contract in code; `src/facts/history.js`, `src/facts/coverage.js` and `src/facts/deps.js` are the three measurements (`history` has `--origins` to chase the first introducer and classifies by message + deleted-line + diff, **on by default since `euthyna audit`**; `coverage.js` reads c8/V8 JSON, coverage.py JSON (format 3) **and Go's text `-coverprofile`** (the latter needs `--source `: the profile carries no function names, so the producer parses top-level `func` ranges from the gofmt'd source; a file absent from the profile is notEvaluated, never "never executed"); `deps.js` reads package-lock.json / Cargo.lock / go.mod, **located from the repository root only** — a monorepo sub-package lock is measured only by `deps --lockfile `, and `audit` does not notice its absence, which `docs/fact-contract.md` §6.3 records with the numbers from the probe). `audit` compares the manifest it measured with the one at `--head` and records `manifest-not-head` when the content differs or the file is absent there. `euthyna audit` (0.4.0, `src/cli.js`) runs history + the full dependency surface in one command — the single entry point the CI recipe and the skill both describe. `src/gate.js` is the six-gate report validator behind `euthyna gate <报告> [--verify] [--allow-exec]` — it parses the skill's 裁定格式, downgrades findings that lack evidence/reproduce or contradict their verdict, and executes nothing but `git` unless the caller consents to interpreters | -| `test/` | 246 tests via `node --test`, no third-party framework. The pass/skip split is keyed on **tar flavour, not the OS**: `test/bench.test.js` skips `every exploit expectation holds` unless `tar --version` reports bsdtar. Measured on `cdece0f`: ubuntu 245 pass + 1 skip (its log line: *the exploit ground truth is defined against bsdtar, not "tar (GNU tar) 1.35"*), windows 246 pass + 0 skip — and this machine's Git Bash PATH puts `D:\Git\usr\bin\tar.exe` (GNU tar 1.35) ahead of System32's `tar.exe`, so a local run reports 245 + 1 even though it is Windows. **The history tests build real git repositories rather than mocking**; the coverage tests carry fixtures shaped like c8, coverage.py and Go profile output (`coverage.test.js` + `coverage-go.test.js`); the deps tests carry fixture lockfiles (npm v1/v2/v3, Cargo.lock, go.mod); the contract tests carry the shell-quoting and render-boundary regressions from PR #1; `git.test.js` carries the error-path quoting and stderr-sanitization regressions from issue #2; `gate.test.js` pins the six-gate validator (including that `--verify` does not run an interpreter command without `--allow-exec`); `cli.test.js` pins the `audit` command, its exit codes, and the manifest-version declaration — content against `--head`, so a PR merge commit whose lockfile is byte-identical to `--head`'s declares nothing while a gitignored lockfile declares itself; the decision itself is an exported pure function so the fail-closed branch (`manifest-unverified`) is reachable from a test; `skill.test.js` pins the written discipline (the six gates, the three verdicts, the `euthyna gate` reference) so weakening the skill turns CI red; `plugin.test.js`, `claude-plugin.test.js` and `hermes-plugin.test.js` pin the three manifest surfaces; `manifest-version.test.js` pins `package.json` against the three plugin manifests (0.5.0 shipped them a release behind and nothing compared them); `i18n.test.js` pins language rendering and report parsing (zh/en); `skill-mirror.test.js` compares the two skill editions against each other — file set, heading sequence (ATX and Setext), fenced blocks, table shapes, and the `euthyna ` / `--flag` identifier sets — so an unmirrored discipline change goes red instead of relying on memory, `docs-pair.test.js` holds `docs/` to the same rule (every document paired, and each pair equal on structure and identifiers — not on prose fidelity), and `markdown-structure.test.js` pins the shared scanner itself with seven crafted-document cases, because a legal Markdown form it cannot read makes any comparison pass on that form (neither guard compares word counts, which a translation legitimately needs room for). **The suite count is written down once, in this row**: README's quickstart carried its own number (207) and was 39 tests stale, so it now says only what `npm test` does, and `package.json` defines that as `node --test` | +| `bin/` + `src/` | **Fact producers** (zero-dependency Node CLI). `src/contract.js` is the contract in code; `src/facts/history.js`, `src/facts/coverage.js` and `src/facts/deps.js` are the three measurements (`history` has `--origins` to chase the first introducer and classifies by message + deleted-line + diff, **on by default since `euthyna audit`**; `coverage.js` reads c8/V8 JSON, coverage.py JSON (format 3), Go's text `-coverprofile` **and lcov tracefiles** (the Go profile needs `--source `: the profile carries no function names, so the producer parses top-level `func` ranges from the gofmt'd source; a file absent from the profile is notEvaluated, never "never executed"; the lcov path needs no `--source` because `FN:` carries the names — they are mangled, and the producer matches a `module::function` query against the `` tail segments of Rust v0 mangling (what `-C instrument-coverage` implies), since no demangler exists on this machine — each segment is anchored at a boundary and never inside a longer `` pair, so a query for `get` does not answer for `cfg_3get`; see `docs/fact-contract.md` §6.2 for the measured record, including that inlining was measured **not** to flatten a called function to 0); `deps.js` reads package-lock.json / Cargo.lock / go.mod, **located from the repository root only** — a monorepo sub-package lock is measured only by `deps --lockfile `, and `audit` does not notice its absence, which `docs/fact-contract.md` §6.3 records with the numbers from the probe). `audit` compares the manifest it measured with the one at `--head` and records `manifest-not-head` when the content differs or the file is absent there. `euthyna audit` (0.4.0, `src/cli.js`) runs history + the full dependency surface in one command — the single entry point the CI recipe and the skill both describe. `src/gate.js` is the six-gate report validator behind `euthyna gate <报告> [--verify] [--allow-exec]` — it parses the skill's 裁定格式, downgrades findings that lack evidence/reproduce or contradict their verdict, and executes nothing but `git` unless the caller consents to interpreters | +| `test/` | 283 tests via `node --test`, no third-party framework. The pass/skip split is keyed on **tar flavour, not the OS**: `test/bench.test.js` skips `every exploit expectation holds` unless `tar --version` reports bsdtar. Measured on `cdece0f`: ubuntu 245 pass + 1 skip (its log line: *the exploit ground truth is defined against bsdtar, not "tar (GNU tar) 1.35"*), windows 246 pass + 0 skip — and this machine's Git Bash PATH puts `D:\Git\usr\bin\tar.exe` (GNU tar 1.35) ahead of System32's `tar.exe`, so a local run **on `cdece0f`** reported 245 + 1 even though it is Windows (measured again 2026-10-12 on the lcov branch: pwsh 283 pass / 0 skip, Git Bash 282 pass / 1 skip, the difference being that one case — the 245/246 pair above is the `cdece0f` record and 283/282 is the current count, so read that scope with the number). **The history tests build real git repositories rather than mocking**; the coverage tests carry fixtures shaped like c8, coverage.py, Go profile and lcov tracefile output (`coverage.test.js` + `coverage-go.test.js` + `coverage-lcov.test.js`, the lcov fixtures using mangled names copied from a real `llvm-cov export --format=lcov` run); the deps tests carry fixture lockfiles (npm v1/v2/v3, Cargo.lock, go.mod); the contract tests carry the shell-quoting and render-boundary regressions from PR #1; `git.test.js` carries the error-path quoting and stderr-sanitization regressions from issue #2; `gate.test.js` pins the six-gate validator (including that `--verify` does not run an interpreter command without `--allow-exec`); `cli.test.js` pins the `audit` command, its exit codes, and the manifest-version declaration — content against `--head`, so a PR merge commit whose lockfile is byte-identical to `--head`'s declares nothing while a gitignored lockfile declares itself; the decision itself is an exported pure function so the fail-closed branch (`manifest-unverified`) is reachable from a test; `skill.test.js` pins the written discipline (the six gates, the three verdicts, the `euthyna gate` reference) so weakening the skill turns CI red; `plugin.test.js`, `claude-plugin.test.js` and `hermes-plugin.test.js` pin the three manifest surfaces; `manifest-version.test.js` pins `package.json` against the three plugin manifests (0.5.0 shipped them a release behind and nothing compared them); `i18n.test.js` pins language rendering and report parsing (zh/en); `skill-mirror.test.js` compares the two skill editions against each other — file set, heading sequence (ATX and Setext), fenced blocks, table shapes, and the `euthyna ` / `--flag` identifier sets — so an unmirrored discipline change goes red instead of relying on memory, `docs-pair.test.js` holds `docs/` to the same rule (every document paired, and each pair equal on structure and identifiers — not on prose fidelity), and `markdown-structure.test.js` pins the shared scanner itself with seven crafted-document cases, because a legal Markdown form it cannot read makes any comparison pass on that form (neither guard compares word counts, which a translation legitimately needs room for). **The suite count is written down once, in this row**: README's quickstart carried its own number (207) and was 39 tests stale, so it now says only what `npm test` does, and `package.json` defines that as `node --test` | | `.agents/skills/euthyna/` | **The skill.** Doubles as source and as a project skill root (rank 200), so it is live in this workspace without a restart | | `bench/` | **The recall benchmark.** `exploits.js` establishes ground truth by execution (18 cases, 8 near-neighbour pairs); `prepare-blind.js` produces answer-free copies with per-round shuffled ids; `adjudicate.js` closes the loop end to end (blind tree → one adjudicator process per case → `collect-verdicts.js` machine validation → `score.js`), with a deterministic `golden` mode (ground truth written into reports — plumbing self-check only, never a real round) for CI; `collect-verdicts.js` machine-validates reports and extracts verdicts without the orchestrator reading report bodies; `score.js` computes the confusion matrix and the pair view; `perf.js` is the `history` scaling baseline. The verified-platform matrix (bsdtar vs GNU tar, p6 skip) lives in `bench/README.md`. Results: `bench/RESULTS.md` (round 1), `bench/RESULTS-round2.md` (round 2: three independent runs per case, zero flips), `bench/RESULTS-round3.md` (round 3: 18 cases, 54 adjudications, two fixture defects caught by adjudicators), `bench/RESULTS-round4.md` (round 4: 54/54, p6b v3 confirmed by first blind adjudication, cross-model run executed — zero flips across two model families); protocols in `bench/README.md`, design pre-registrations in `bench/DESIGN-round3.md` and `bench/DESIGN-round4.md` | | `docs/positioning.md` + `-zh` | Competitive analysis across the DSH catalog, including three claims that were tested and refuted | diff --git a/README.md b/README.md index 71cb51d..92aa1c6 100644 --- a/README.md +++ b/README.md @@ -253,10 +253,18 @@ This project tries to be explicit about the difference. Current state: unchanged on a Go repository ([nektos/act](https://github.com/nektos/act), where the deleted lines of a security fix were traced back to the commit that introduced them) and on a Rust one ([rustls](https://github.com/rustls/rustls), whose fix range yielded 43 provenance facts). -- **`coverage` on real output in three formats** — c8/V8 JSON, classic istanbul (jest/nyc, same - fnMap/f shape) and coverage.py JSON (format 3) — distinguishing all three states correctly, and - refusing anything that is not a recognizable coverage report instead of answering "symbol not - located" against it. +- **`coverage` on real output in five formats** — c8/V8 JSON, classic istanbul (jest/nyc, same + fnMap/f shape), coverage.py JSON (format 3), Go's text `-coverprofile`, and lcov tracefiles + (`llvm-cov export --format=lcov`, which is how cargo-llvm-cov and cargo-tarpaulin emit Rust + coverage) — distinguishing all three + states correctly, and refusing anything that is not a recognizable coverage report instead of + answering "symbol not located" against it. The lcov path was measured against a probe crate + built with `-C instrument-coverage`: an uncalled function reports a count of 0, a called one + reports a count above it, and inlining was measured **not** to flatten a called function to 0. + What it cannot do is demangle — this machine has no working demangler, so a query is matched + against the `` tail segments of the mangled name; a bare `dup` matches + every same-named function in the file, each reported separately, and a query never answers for + a function that merely contains the queried name (`get` does not match `cfg_3get`). - **`deps` against real manifests** — resolved versions reported with a line-level evidence pointer into the lockfile, and absent dependencies reported as established absences rather than silent skips. Verified against a populated npm v3 lockfile, a real `go.mod` (the *declared* diff --git a/docs/case-study-rust-zh.md b/docs/case-study-rust-zh.md index b1250ac..a17952d 100644 --- a/docs/case-study-rust-zh.md +++ b/docs/case-study-rust-zh.md @@ -252,6 +252,13 @@ worst verdict per crate: {"unaffected":52,"never-patched":3,"affected":4,"unknow 因此第 4 条裁定的第 3、4 道门禁是"未评估"而不是"fail"：**缺覆盖率事实时，那道门禁记未评估， 不得当作驳回的承担者**——这是上一轮 PR #21 刚收敛的规则，本轮第一次在真实数据上用到它。 +> **后续（2026-10-12）：这个缺口已经补上了。** 上面的记录是本轮当时的实况，不改写； +> 但 `src/facts/coverage.js` 现在认识 lcov tracefile 了，本产出器的格式清单从三种变成四种。 +> 补的时候用的正是一个探针 crate（12 个函数、2 个文件）走 +> `RUSTFLAGS="-C instrument-coverage"` + `cargo test` + `llvm-cov export --format=lcov` 这条路， +> 而没有依赖 `cargo-llvm-cov`（本机仍然没有）。**本条不影响上面那条"当时没有"的结论**： +> 它记录的是 0.6.0 那一轮在 rustls 上跑出来的结果。 + --- ## 8. 怎么复现 diff --git a/docs/case-study-rust.md b/docs/case-study-rust.md index 3973278..734592e 100644 --- a/docs/case-study-rust.md +++ b/docs/case-study-rust.md @@ -278,6 +278,14 @@ So gates 3 and 4 of the fourth verdict are "not evaluated" rather than "fail": * fact is missing, that gate is recorded as not evaluated and must not carry a dismissal** — the rule PR #21 had just converged, used here for the first time on real data. +> **Follow-up (2026-10-12): that gap has since been closed.** The record above is what was true in +> that round and is not rewritten; but `src/facts/coverage.js` now reads lcov tracefiles, so this +> producer's format list has gone from three to four. It was closed with a probe crate (12 +> functions, 2 files) driven through `RUSTFLAGS="-C instrument-coverage"` + `cargo test` + +> `llvm-cov export --format=lcov`, without depending on `cargo-llvm-cov` (still absent here). **This +> does not change the "was not there then" conclusion above**: it records what the 0.6.0 round +> measured on rustls. + --- ## 8. How to reproduce diff --git a/docs/fact-contract-zh.md b/docs/fact-contract-zh.md index 2a537ec..72f008f 100644 --- a/docs/fact-contract-zh.md +++ b/docs/fact-contract-zh.md @@ -305,6 +305,49 @@ return checkPermission(user, 'never'); // 从未执行，但 c8 报 hits=1 —— **不是 istanbul 的 if/else 模型**，而是 V8 块范围换了个标签（`v8-to-istanbul/lib/branch.js:21-27`）， 但它**带完整 start/end 行列**，所以能做真正的范围包含判断 +#### 已实测的 lcov 输出形状（Rust / llvm-cov，2026-10-12） + +`llvm-cov export --format=lcov` 的 tracefile 是本产出器读的第五种格式——前四种是 c8/V8 JSON、 +classic istanbul、coverage.py 格式 3 与 Go 的文本 profile；cargo-llvm-cov 与 cargo-tarpaulin +都产它，gcov 与 llvm-cov 也都产这个格式。**格式的覆盖面比符号匹配更宽**：查询只能对上路径段 +按 `<十进制长度><名字>` 编码的名字，所以纯 C 目标的 `FN:10,main` 永远匹配不上——实测 +`--symbol main` 对着 needle `4main` 报 "could not locate"。实测样本是一个两文件探针 crate—— +14 处函数声明、15 条 `FN:`（一个泛型函数在同一行单态化成两个符号，`FNF:` 把它们记作 12 与 +2）——`RUSTFLAGS="-C instrument-coverage"` + `cargo test` + `llvm-cov export`： + +- 每条记录以 `SF:<绝对路径>` 开头、`end_of_record` 结束；Windows 上是带反斜杠的绝对路径 +- **`FN:` 与 `FNDA:` 是两段分开的**（先全部 `FN:<行>,`，再全部 + `FNDA:<计数>,`），不是相邻配对。按行读流时不能假设它们成对出现 +- 符号名是 **Rust v0 mangling**（`_RNvCs…10lcov_probe12never_called`）——`-C instrument-coverage` + 蕴含的就是它（`rustc -C help`：it "implies -C symbol-mangling-version=v0"）；更老的 + Itanium 形 `_ZN…17h…E` 是裸 `rustc` 产的名字，不是这条路径实测过的形状。尾段 + `<十进制长度><名字>` 是路径段的编码。**本机没有任何可用 demangler**：`llvm-cov --Xdemangler` + 试过 `rust`/`rustfilt`/`llvm-cxxfilt`/`c++filt` 四个名字，全部报 + `could not find the demangler`。所以对齐只能在产出器侧做：把查询按 `::` 切段， + 每段按 `<字节长度><段名>` 锚定匹配（名字开头、mangling 自己的 `_` 分隔符之后、或已证明的 + pair 边界），且**绝不落在更长的 pair 内部**——最后这半句就是「查 `get` 不会答到 `cfg_3get`」 + 的原因：`_RNvCsi8SMnIpCoEi_13collide_probe8cfg_3get` 整体读成一个完整的 pair，于是 `3get` + 落在标识符内部。本分支早先的版本会对 `cfg_3get` 报 "Symbol get was not invoked even once"， + 而这对真正被调用过的 `get` 来说是假的。`nested::dup` 只命中 `6nested3dup`； + **裸 `dup` 会同时命中外层模块里的同名函数**——这是刻意镜像 c8 对裸方法名的行为 + （每个命中各自报一条，各自带自己的 mangled 名），不是缺陷。**带限定**的查询是子序列判定、 + 不是路径精确判定，所以 `lcov_probe::dup` 也会答 `lcov_probe::nested::dup`；每条事实都写明 + 自己针对哪个符号，`::` 查询的语句用那个 mangled 名，而不是把模式当符号来陈述 +- **同一行可以有多条 `FN:`**（泛型单态化：同一个 `generic_fn` 用 `i32` 和 `&str` 各调一次 + 就是同一行 48 上的两条）。一个泛型实例化两次 = 两条事实 +- 同一 `SF:` 出现多条记录、或同一符号多条 `FNDA:` 时**计数求和**。求和是唯一不会把非零 + 变零的合并方式，对唯一可行动的结论（计数为 0）安全 +- 文件不在 tracefile 里报**「未评估」**，与 c8 的「文件不在数据里 = 从未执行」**刻意相反**： + llvm-cov/gcov 只为被插桩的目标产记录，`--sources` 还会再过滤一层 +- `FN:` 有而 `FNDA:` 无 → **「未评估」，绝不默认成 0** + +**内联不会制造假的「从未调用」（实测，反向验证过）。** `count === 0 ⇒ 从未调用` 这条 +唯一可行动的推论，唯一的致命反例是「函数被调用了、但被完全内联掉、于是计数为 0」。 +在一个专门的探针 crate 上实测 release 构建 + `-C instrument-coverage`： +`#[inline(always)]` 的函数被调用时仍报 `FNDA:1`（`#[inline(never)]` 的对照同样报 1）， +即**没有观测到内联把已调用的函数压成 0**。这条只在这个范围内成立——它测的是 +llvm-cov 插桩下的行为，不是「任何构建配置下都安全」的普遍保证。 + #### 消费端必须处理的失败模式 | 情况 | 表现 | 要求 | @@ -313,7 +356,7 @@ return checkPermission(user, 'never'); // 从未执行，但 c8 报 hits=1 | c8 默认排除规则 | `test/**`、`**/*.test.js`、`**/__tests__/**` 被排掉 | 生产调用方落这些 glob 会**静默消失** | | V8 块范围负列号 | 实测出现 `L9:-1-L10:0` | 朴素列比较会崩 | | 模块顶层调用点 | 只在 `branchMap` 里（`fnMap` 过滤掉 `functionName===""`） | 不能只查 `fnMap` | -| 根本不是覆盖率文件 | 既无 `fnMap`/`f`（c8/istanbul）也无 `functions`（coverage.py） | 报 **notEvaluated**——喂给定位器会一本正经地答"符号未定位" | +| 根本不是覆盖率文件 | 既无 `fnMap`/`f`（c8/istanbul）也无 `functions`（coverage.py），也不是 Go profile 或 lcov tracefile | 报 **notEvaluated**——喂给定位器会一本正经地答"符号未定位" | | 符号被改名 | 新名字不在 `fnMap` 里 | `unknown`，**绝不**「已确证从未被调用」 | #### 其他语言：比 JS 更差，不要外推 @@ -324,7 +367,7 @@ return checkPermission(user, 'never'); // 从未执行，但 c8 报 hits=1 | Python (coverage.py) | 行 + 分支弧 | ❌ 短路对它完全隐形 | | **Go** | 基本块 | ❌ **比 V8 更差**——实测 `return Strict && Check(...)` 里 `Check` 从未被调用， Go 连块都没切，**连证伪都做不到** | -| Rust (llvm region) | LLVM region | ⚠️ 部分 | +| Rust (llvm region) | LLVM region | ✅ 能**证伪函数级**（lcov 的 `FN:`/`FNDA:` 给每函数的调用计数，2026-10-12 实测：从未调用的报 0、被调用的报 1，含 `#[inline(always)]`）；调用点级仍是 ⚠️ 部分 | #### 与 prior art 的关系 diff --git a/docs/fact-contract.md b/docs/fact-contract.md index 4946e63..472a38d 100644 --- a/docs/fact-contract.md +++ b/docs/fact-contract.md @@ -305,6 +305,60 @@ Hence the hard constraints at the contract level: — **not istanbul's if/else model**, but V8 block ranges under a different label (`v8-to-istanbul/lib/branch.js:21-27`), and it **carries complete start/end line and column**, so genuine range-containment checks are possible +#### The measured lcov output shape (Rust / llvm-cov, 2026-10-12) + +An `llvm-cov export --format=lcov` tracefile is the fifth format this producer reads — after +c8/V8 JSON, classic istanbul, coverage.py format 3 and Go's text profile; cargo-llvm-cov and +cargo-tarpaulin emit it, and gcov and llvm-cov both produce the format. **The format is wider +than the symbol matching**: a query resolves only against a name whose path segments are encoded +as ``, so a plain C target's `FN:10,main` never matches — measured, +`--symbol main` answers "could not locate" against the needle `4main`. The measured +sample is a two-file probe crate — 14 function declarations, 15 `FN:` entries, because one +generic function is monomorphised onto two symbols at the same line (`FNF:` counts them as 12 +and 2) — built with `RUSTFLAGS="-C instrument-coverage"`, `cargo test`, then `llvm-cov export`: + +- Each record opens with `SF:` and closes with `end_of_record`; on Windows the + path is absolute and backslash-separated +- **`FN:` and `FNDA:` come in two separate runs** (every `FN:,` first, then every + `FNDA:,`), not as adjacent pairs. A line-stream parser must not assume pairing +- The symbol names are **Rust v0 mangling** (`_RNvCs…10lcov_probe12never_called`) — which is what + `-C instrument-coverage` implies (`rustc -C help`: it "implies -C symbol-mangling-version=v0"); + the older Itanium-style `_ZN…17h…E` name is what a bare `rustc` emits, and is not the shape this + path was measured against. Tail segments encode a path segment as ``. + **There is no usable demangler on this machine**: `llvm-cov --Xdemangler` was tried with + `rust`/`rustfilt`/`llvm-cxxfilt`/`c++filt` and all four report `could not find the demangler`. + Alignment therefore happens on the producer side: the query is split on `::` and each segment is + matched as ``, anchored at the name's start, after the mangling's own `_` + separator, or at a proven pair boundary **and never inside a longer pair** — that last clause is + what keeps a query for `get` from answering for `cfg_3get`, since + `_RNvCsi8SMnIpCoEi_13collide_probe8cfg_3get` reads as one complete pair and `3get` therefore sits + inside an identifier. An earlier revision of this branch reported "Symbol get was not invoked + even once" about `cfg_3get` — false about `get`, the function that had been called. + `nested::dup` matches only `6nested3dup`; **a bare `dup` also matches the same-named function in + an outer module** — deliberately mirroring c8's handling of bare method names (each hit becomes + its own fact carrying its own mangled name), not a defect. A **qualified** query is a subsequence + test rather than a path-exact one, so `lcov_probe::dup` also answers for `lcov_probe::nested::dup`; + every fact names the symbol it is about, and a `::` query states that mangled name rather than + the pattern as if it were a symbol +- **The same line may carry several `FN:` entries** (generic monomorphisation: one `generic_fn` + called once with `i32` and once with `&str` yields two entries on line 48). One generic function + instantiated twice is two facts +- When one `SF:` appears in several records, or one symbol in several `FNDA:` lines, the counts are + **summed**. Summing is the only merge that cannot turn a non-zero into zero, which keeps the one + actionable conclusion (a count of 0) safe +- A file absent from the tracefile is reported as **"not evaluated"**, **deliberately the opposite** + of c8's "file absent from the data = never executed": llvm-cov/gcov only emit records for + instrumented targets, and `--sources` filters again +- An `FN:` with no `FNDA:` is **"not evaluated", never a default of 0** + +**Inlining does not manufacture a false "never invoked" (measured, and verified in both +directions).** The one fatal counterexample to `count === 0 ⇒ never invoked` would be a function +that was called but fully inlined, leaving a count of 0. Measured on a dedicated probe crate with a +release build under `-C instrument-coverage`: an `#[inline(always)]` function that is called still +reports `FNDA:1` (the `#[inline(never)]` control reports 1 as well), so **no case was observed where +inlining flattened a called function to 0**. That holds only within this scope — it measures +llvm-cov-instrumented behaviour, not a universal guarantee across every build configuration. + #### Failure modes a consumer must handle | Situation | Symptom | Requirement | @@ -313,7 +367,7 @@ Hence the hard constraints at the contract level: | c8's default exclude rules | `test/**`, `**/*.test.js`, `**/__tests__/**` are excluded | Production callers that fall under these globs **silently disappear** | | Negative column numbers in V8 block ranges | `L9:-1-L10:0` seen in real testing | Naive column comparison breaks | | Module top-level call sites | Present only in `branchMap` (`fnMap` filters out `functionName===""`) | You cannot query `fnMap` alone | -| A file that is not coverage data at all | Has neither `fnMap`/`f` (c8/istanbul) nor `functions` (coverage.py) | Report as *not evaluated* — feeding it to the locator would answer "symbol not located" with a straight face | +| A file that is not coverage data at all | Has neither `fnMap`/`f` (c8/istanbul) nor `functions` (coverage.py), and is not a Go profile or an lcov tracefile | Report as *not evaluated* — feeding it to the locator would answer "symbol not located" with a straight face | | A symbol that was renamed | Not in `fnMap` under the new name | `unknown`, **never** "established as never invoked" | #### Other languages: worse than JS — do not extrapolate @@ -324,7 +378,7 @@ Hence the hard constraints at the contract level: | Python (coverage.py) | Lines + arcs | ❌ Short circuits are completely invisible to it | | **Go** | Basic blocks | ❌ **Worse than V8** — in testing, `return Strict && Check(...)` never called `Check`; Go does not even split the block, **so it cannot even falsify** | -| Rust (llvm region) | LLVM region | ⚠️ Partial | +| Rust (llvm region) | LLVM region | ✅ **Function-level falsification works** (lcov's `FN:`/`FNDA:` carry a per-function invocation count; measured 2026-10-12: never-invoked reports 0, invoked reports 1, including `#[inline(always)]`); call-site level remains ⚠️ Partial | #### Relation to prior art diff --git a/src/cli.js b/src/cli.js index ff00212..a14cd88 100644 --- a/src/cli.js +++ b/src/cli.js @@ -89,11 +89,12 @@ euthyna —— 给 AI 编码 agent 用的确定性事实产出器 --no-origins 关闭来源追溯；默认用 git log -S 把删除行归属到「最初引入」 该内容的提交（有探针上限，比纯 blame 慢） coverage 某个符号在测试运行中到底有没有被调用过 - --coverage 覆盖率数据文件，c8 的 coverage-final.json + --coverage 覆盖率数据文件，按形状自动识别 c8 / istanbul、 + coverage.py JSON、Go -coverprofile 文本、lcov tracefile --symbol 要查询的符号名，可重复 --file 可选，限定到某个文件 --source 可选，Go profile（go test -coverprofile）必需：模块根目录， - 用于把行区间映射回函数（profile 不含函数名） + 用于把行区间映射回函数（profile 不含函数名）；其余格式不需要 deps 某个依赖在 lockfile 里被锁定/声明成什么版本（供应链声明的裁决依据） --repo 可选，依赖清单所在目录（默认当前目录，自动检测） --lockfile 可选，显式指定清单文件（支持 package-lock.json / Cargo.lock / go.mod） @@ -146,12 +147,14 @@ Commands: lines to the commit that FIRST introduced their content (probe-capped, slower than plain blame) coverage was this symbol ever actually invoked by a test run? - --coverage the coverage data file, c8's coverage-final.json + --coverage the coverage data file; identified by shape as c8 / istanbul, + coverage.py JSON, a Go -coverprofile, or an lcov tracefile --symbol symbol name(s) to query, repeatable --file optional, restrict to one file --source optional, required for Go profiles (go test -coverprofile): the module root, used to map line ranges back to - functions (the profile carries no function names) + functions (the profile carries no function names); other + formats do not need it deps what version does the lockfile pin or declare for a dependency? (the basis for supply-chain claims) --repo optional, directory containing the manifest (default cwd, auto-detected) --lockfile optional, explicit manifest path (package-lock.json / Cargo.lock / go.mod) diff --git a/src/facts/coverage.js b/src/facts/coverage.js index d66cebd..f27a847 100644 --- a/src/facts/coverage.js +++ b/src/facts/coverage.js @@ -63,7 +63,7 @@ function reproduceCommand({ coverageFile, symbol, file, source }) { /** * Read a coverage report. * - * Three formats are accepted, all identified by shape rather than by filename: + * Five formats are accepted, all identified by shape rather than by filename: * * 1. c8 / v8-to-istanbul `coverage-final.json` — per-file keys are * path/all/statementMap/s/branchMap/b/fnMap/f. There is no istanbul `hash`, @@ -88,6 +88,12 @@ function reproduceCommand({ coverageFile, symbol, file, source }) { * parsed from the gofmt'd source under --source (module root). A file * absent from the profile is *not* evidence of "never executed" (the * package may not be in the test binary), so absence is notEvaluated. + * 5. lcov tracefiles (`llvm-cov export --format=lcov`, gcov `.info`, + * cargo-llvm-cov) — text records, one per source file, opened by `SF:` and + * closed by `end_of_record`, with `FN:,` / `FNDA:,` + * pairs inside. Function names **are** present, but mangled; see + * parseLcov() for how a query is aligned with them and why `--source` is + * not needed on this path. * * Anything that matches none of these shapes is **not a coverage report this * producer can read**, and is reported as notEvaluated — never fed through the @@ -105,6 +111,17 @@ export async function loadCoverage(coverageFile) { } return parsed; } + if (isLcovTracefile(raw)) { + const parsed = parseLcov(raw); + if (!parsed) { + throw new Error( + 'the file looks like an lcov tracefile ("SF:" records closed by "end_of_record") but holds a line that is not ' + + 'an lcov "key:value" line, a function line outside any record, an unclosed record, or an FN value that is not ' + + '"," — refusing to interpret it as coverage' + ); + } + return parsed; + } return JSON.parse(raw); } @@ -249,6 +266,450 @@ export function goFunctionRanges(text) { return funcs; } +/** + * lcov tracefiles (`llvm-cov export --format=lcov`, gcov `.info`, + * cargo-llvm-cov) are text records, one per source file: + * + * SF: + * FN:, + * FNDA:, + * FNF:12 + * FNH:11 + * DA:, + * BRF:0 + * BRH:0 + * LF:52 + * LH:46 + * end_of_record + * + * Unlike a Go profile these records **do** carry function names, so a symbol + * query needs no source tree — but the names are mangled, which is what + * lcovSymbolMatches() is for. + * + * Two things this reader refuses rather than guesses: + * + * - an `FN:` with no matching `FNDA:` is "no count", not "count 0". Defaulting + * it to zero would manufacture a "never invoked" out of a missing line, + * which is the one direction this producer must never get wrong. + * - the newer `FNL:` / `FNA:` / `FNAB:` encodings are recognised and rejected. + * No sample of them could be produced on this machine, and a function table + * read out of a shape nobody has seen is the same failure + * detectCoverageFormat() refuses — half a shape treated as coverage. (Their + * spelling is taken from the lcov specification, not from an observed file.) + */ +function isLcovTracefile(raw) { + const first = String(raw) + .split(/\r?\n/) + .find(line => line.trim()); + return /^(SF|TN):/.test((first ?? '').trim()); +} + +const LCOV_ENTRY_RE = /^([A-Za-z_]+):(.*)$/; +const LCOV_FN_RE = /^(\d+),(.+)$/; + +/** + * Parse an lcov tracefile into + * `{ __lcov, files: Map }> }`, + * or null if any line is not one this reader can account for. + * + * `count` is null when a function is declared without an FNDA line: the parser + * never substitutes a number for a count that is not in the file. + * + * Two records for the same `SF:` (a split tracefile, or one file instrumented + * twice) are merged by summing counts. Summing is the one merge that cannot + * turn a nonzero count into zero, and zero is the only conclusion this producer + * draws — so the merge is safe in exactly the direction that matters. A name + * whose count is missing on either side stays null. + * + * DA/BRDA/FNF/FNH/LF/LH/BRF/BRH/TN/VER are recognised and ignored: the fact + * model here is function-level, and refusing a file over a line we never read + * would refuse files that answer the query correctly. + */ +export function parseLcov(raw) { + const lines = String(raw).split(/\r?\n/); + const files = new Map(); + let fnLines = null; + let fnda = null; + let currentPath = null; + + const closeRecord = () => { + const functions = new Map(); + for (const [symbol, line] of fnLines) { + functions.set(symbol, { line, count: fnda.has(symbol) ? fnda.get(symbol) : null }); + } + // An FNDA without an FN still carries a count, and a count is enough to say + // "never invoked"; only its line is unknown. + for (const [symbol, count] of fnda) { + if (!functions.has(symbol)) functions.set(symbol, { line: null, count }); + } + const previous = files.get(currentPath); + if (previous) { + for (const [symbol, fn] of functions) { + const prev = previous.functions.get(symbol); + if (!prev) { + previous.functions.set(symbol, fn); + continue; + } + previous.functions.set(symbol, { + line: prev.line ?? fn.line, + count: prev.count === null || fn.count === null ? null : prev.count + fn.count + }); + } + } else { + files.set(currentPath, { functions }); + } + fnLines = null; + fnda = null; + currentPath = null; + }; + + for (const line of lines) { + if (!line.trim()) continue; + if (line.trim() === 'end_of_record') { + if (currentPath === null) return null; + closeRecord(); + continue; + } + const entry = LCOV_ENTRY_RE.exec(line); + if (!entry) return null; + const [, key, value] = entry; + + // `TN:` names the test run and legally precedes a record (it is also what + // isLcovTracefile keys on when a file has no SF line yet), so it is ignored + // anywhere rather than refused outside a record. + if (key === 'TN') continue; + + if (key === 'SF') { + if (currentPath !== null) return null; // a record was never closed + currentPath = value; + fnLines = new Map(); + fnda = new Map(); + continue; + } + if (currentPath === null) return null; // any other key outside a record + if (key === 'FN' || key === 'FNDA') { + const pair = LCOV_FN_RE.exec(value); + if (!pair) return null; + const [, first, symbol] = pair; + if (key === 'FN') { + if (!fnLines.has(symbol)) fnLines.set(symbol, Number(first)); + } else { + fnda.set(symbol, (fnda.get(symbol) ?? 0) + Number(first)); + } + continue; + } + if (key === 'FNL' || key === 'FNA' || key === 'FNAB') return null; + } + + if (currentPath !== null) return null; // unclosed record at end of file + if (files.size === 0) return null; // no SF: record at all + return { __lcov: true, files }; +} + +/** + * The needle for one query segment: Rust's v0 mangling and Itanium both + * encode an identifier as ``, so `dup` is + * looked up as `3dup`. + */ +export function lcovSymbolNeedle(segments) { + return segments.map(segment => `${Buffer.byteLength(segment, 'utf8')}${segment}`).join(''); +} + +/** + * True when `name.slice(k, end)` is a complete `` pair — the + * proof that `end` is where a fresh path segment begins. + */ +function lcovPairEndsAt(name, end) { + for (let k = 0; k < end; k++) { + if (!/\d/.test(name[k])) continue; + if (k > 0 && /\d/.test(name[k - 1])) continue; // only the start of a digit run + let digits = k; + while (digits < end && /\d/.test(name[digits])) digits++; + if (digits >= end) continue; + if (digits + Number(name.slice(k, digits)) === end) return true; + } + return false; +} + +/** + * True when the identifier `name.slice(idStart, idEnd)` sits strictly inside a + * longer, complete `` pair — a reading of the name in which the + * identifier is part of a *longer* one, so a needle landing there is not + * standing at a segment boundary. + * + * Measured, and the reason this exists: in + * `_RNvCsi8SMnIpCoEi_13collide_probe8cfg_3get` the character before `3get` is + * `_`, which the separator rule accepts, but `8cfg_3get` is itself a complete + * pair (`8` is the length of `cfg_3get`), so `3get` is inside an identifier and + * a query for `get` must not answer for it. + */ +function lcovInsideLongerPair(name, idStart, idEnd) { + for (let k = 0; k < idStart; k++) { + if (!/\d/.test(name[k])) continue; + if (k > 0 && /\d/.test(name[k - 1])) continue; // only the start of a digit run + let digits = k; + while (digits < name.length && /\d/.test(name[digits])) digits++; + if (digits >= name.length) continue; + const start = digits; + const end = start + Number(name.slice(k, digits)); + if (start >= idEnd || end > name.length) continue; + if (start < idStart && end >= idEnd) return true; + } + return false; +} + +/** + * Does a mangled name answer for the query path `segments`? + * + * `true` when every segment appears, in order, each one anchored: at the start + * of the name, right after the mangling's own separators (`_`, as in + * `NtB2_7Counter4bump`), or at a proven `` pair boundary — and + * not strictly inside a longer pair (see lcovInsideLongerPair). The segments are + * matched independently rather than as one concatenated needle because a + * mangling may insert both a disambiguator and further path components between + * them: a `#[cfg(test)]` module comes out as `...5testss_14exercises_some`, and + * the path of a trait method carries its `Self` type + * (`NtB4_4GearNtB4_6Widget5label`), so a concatenated needle would miss both + * `tests::exercises_some` and `Gear::label` outright — measured, by running the + * contiguous form against the real tracefiles. + * + * What this buys: the digits before an identifier are its *byte* length, so + * requiring the digits to be there, and to be a real length prefix, is what + * keeps a query for `dup` from matching a function whose own name merely ends + * with those characters (`7xyz3dup` is one segment, and the character before + * `3dup` is `z`, which proves no pair ends there). + * + * The boundary test's fourth alternative — any character outside `[A-Za-z0-9_]` + * — is broader than anything measured needs: every character of the 15 `FN:` + * symbols in the probe tracefile is inside that class, so the mangling's own + * `_` separator and the length prefixes are the only boundaries the measurement + * actually exercises. It stays permissive on purpose: the observed names give no + * reason to narrow it, and a narrower rule invented without the mangling's + * grammar could reject a legal boundary it cannot recognise. + * + * One residual over-inclusion, recorded rather than hidden: a *qualified* query + * is a subsequence test, not a path-exact one, so `lcov_probe::dup` also + * answers for `lcov_probe::nested::dup`. Each fact names the symbol it is about + * (see collectLcovCoverageFacts), so the reader can tell them apart; narrowing + * the match further would need the mangling's grammar. + * + * A query that is already mangled (`_R`/`_Z`) is matched exactly and never + * decoded: this producer has no demangler, and half-decoding one would be a + * guess about which symbol was asked for. + */ +export function lcovSymbolMatches(segments, name) { + if (segments.length === 0) return false; + if (segments.length === 1 && /^_[RZ]/.test(segments[0])) return name === segments[0]; + + let from = 0; + for (const segment of segments) { + const needle = lcovSymbolNeedle([segment]); + const idDigits = String(Buffer.byteLength(segment, 'utf8')).length; + let found = -1; + for (let at = name.indexOf(needle, from); at !== -1; at = name.indexOf(needle, at + 1)) { + const boundary = + at === 0 || name[at - 1] === '_' || !/[A-Za-z0-9_]/.test(name[at - 1]) || lcovPairEndsAt(name, at); + if (boundary && !lcovInsideLongerPair(name, at + idDigits, at + needle.length)) { + found = at; + break; + } + } + if (found === -1) return false; + from = found + needle.length; + } + return true; +} + +/** + * A query answered from the lcov function table. + * + * No `--source`: the tracefile carries function names, so this path answers the + * question Go's profile cannot answer without the source tree. The trade is + * that the names are mangled and the matching is done on the mangled form (see + * lcovSymbolMatches), and that a symbol which never made it into the + * instrumented build has no source-tree search to fall back on — it comes out + * as "not located" (unknown), never as clean. + */ +async function collectLcovCoverageFacts({ coverage, coverageFile, targets, lang }) { + const t = T(lang); + const facts = []; + const evaluated = []; + const notEvaluated = []; + let counter = 0; + + const entries = [...coverage.files.entries()]; + + if (!entries.some(([, record]) => record.functions.size > 0)) { + // Line coverage cannot answer a symbol question: which lines belong to a + // function is in the source, not in the tracefile. + notEvaluated.push( + notEvaluatedEntry( + KIND.TEST_COVERAGE, + t( + 'lcov tracefile 里没有任何 FN: 行 —— 只有文件与行数据，没有函数名。符号级问题（这个符号被调用过吗）' + + '需要 FN:/FNDA: 记录才能回答，行覆盖回答不了它', + 'The lcov tracefile carries no FN: lines — file and line records only, no function names. ' + + 'A symbol-level question (was this symbol invoked) needs FN:/FNDA: records; line coverage cannot answer it' + ) + ) + ); + return { facts, evaluated, notEvaluated, measured: false }; + } + + if (targets.length === 0) { + notEvaluated.push( + notEvaluatedEntry( + KIND.TEST_COVERAGE, + t( + '没有指定要查询的符号（--symbol）', + 'No symbol was given to look up (--symbol)' + ) + ) + ); + return { facts, evaluated, notEvaluated, measured: false }; + } + + for (const target of targets) { + const scoped = target.file + ? entries.filter(([entryPath]) => matchesTarget(entryPath, target.file)) + : entries; + + if (target.file && scoped.length === 0) { + // Deliberately the opposite of the c8 path above: a c8 report lists the + // files it instrumented, so absence there means "not loaded", while + // llvm-cov and gcov emit records only for what was instrumented and + // --sources filters further — absence here proves nothing about execution. + notEvaluated.push( + notEvaluatedEntry( + KIND.TEST_COVERAGE, + t( + `${target.file} 没有出现在 lcov tracefile 中 —— llvm-cov/gcov 只为被插桩的目标产记录（--sources 还会再过滤一层），` + + '所以「未出现」不能当「未执行」', + `${target.file} does not appear in the lcov tracefile — llvm-cov/gcov emit a record only for what was instrumented ` + + '(and --sources filters further), so "absent" cannot be read as "never executed"' + ) + ) + ); + continue; + } + + const segments = target.symbol + .split('::') + .map(segment => segment.trim()) + .filter(Boolean); + const matches = []; + for (const [entryPath, record] of scoped) { + for (const [mangled, fn] of record.functions) { + if (lcovSymbolMatches(segments, mangled)) matches.push({ entryPath, mangled, ...fn }); + } + } + + if (matches.length === 0) { + const needle = lcovSymbolNeedle(segments); + facts.push( + makeFact({ + id: `coverage-${++counter}`, + kind: KIND.TEST_COVERAGE, + statement: + t( + `未能在覆盖率数据中定位符号 ${target.symbol}`, + `Could not locate symbol ${target.symbol} in the coverage data` + ) + + (target.file ? t(`（限定文件 ${target.file}）`, ` (restricted to file ${target.file})`) : '') + + t( + ` —— lcov 的 FN: 名是 mangled 的，本产出器按段边界匹配 "${needle}"；` + + '泛型实例、被内联的克隆，或写法不同的路径（crate:: 前缀、以 . 分隔）都可能匹配不上，' + + '也可能是它根本没被编译进这次插桩构建', + ` — lcov FN: names are mangled and this producer matches them by segment boundaries against "${needle}"; ` + + 'a generic instantiation, an inlined clone, or a differently spelled path (a crate:: prefix, a "." separator) may fail to match, ' + + 'and the symbol may simply not have been compiled into this instrumented build' + ), + status: STATUS.UNKNOWN, + evidence: { file: target.file ?? t('(未限定文件)', '(unrestricted file)') }, + method: 'static', + detail: { symbol: target.symbol, reason: 'symbol_not_located_in_fnmap' } + }) + ); + continue; + } + + // A query can legitimately hit several symbols — a bare name repeated in two + // modules, or one generic function instantiated twice at the same line. Each + // is reported on its own so no fact claims more than the symbol it names: a + // bare name is a symbol name and can be stated as given, while a `::` path is + // a pattern, so the statement names the symbol the match actually resolved to. + const pathQuery = segments.length > 1; + for (const hit of matches) { + const subject = pathQuery ? hit.mangled : target.symbol; + const qualifier = matches.length > 1 && !pathQuery + ? t( + `（本次查询匹配到 ${matches.length} 个符号；本事实针对 ${hit.mangled}）`, + ` (${matches.length} symbols matched this query; this fact is about ${hit.mangled})` + ) + : ''; + + if (hit.count === null) { + notEvaluated.push( + notEvaluatedEntry( + KIND.TEST_COVERAGE, + t( + `${hit.mangled} 在 tracefile 里有 FN: 行却没有任何 FNDA: 计数 —— 缺失的计数不是 0，这条记录答不了` + + `「${subject} 是否被调用过」`, + `${hit.mangled} has an FN: line in the tracefile but no FNDA: count at all — a missing count is not a zero, ` + + `so this record cannot answer whether ${subject} was invoked` + ) + ) + ); + continue; + } + + const zero = hit.count === 0; + facts.push( + makeFact({ + id: `coverage-${++counter}`, + kind: KIND.TEST_COVERAGE, + statement: zero + ? t( + `符号 ${subject} 在本次测试运行中一次都没有被调用（调用计数为 0）—— 任何依赖它的行为都没有被执行验证`, + `Symbol ${subject} was not invoked even once in this test run (invocation count 0) — ` + + 'no behaviour depending on it was exercised' + ) + qualifier + : t( + `符号 ${subject} 在本次测试运行中至少被调用过 ${hit.count} 次，但被调用**不能**证明任何特定调用点执行过 —— ` + + '本事实只能证伪，不能证实', + `Symbol ${subject} was invoked at least ${hit.count} time(s) in this test run, but being invoked ` + + '**cannot** prove that any particular call site executed — this fact can refute, not confirm' + ) + qualifier, + status: zero ? STATUS.ESTABLISHED : STATUS.UNKNOWN, + evidence: hit.line === null ? { file: hit.entryPath } : { file: hit.entryPath, line: hit.line }, + method: 'command', + command: reproduceCommand({ coverageFile, symbol: target.symbol, file: hit.entryPath }), + detail: { + symbol: target.symbol, + mangled: hit.mangled, + invocationCount: hit.count, + reason: zero ? 'invocation_count_zero' : 'nonzero_count_cannot_prove_call_site_execution' + } + }) + ); + } + } + + // Same rule as the Go path: a query answered only by refusals was not answered + // at all, and "measured" must not let CI read that as clean. + const measured = facts.length > 0; + if (measured) { + evaluated.push({ + kind: KIND.TEST_COVERAGE, + producer: 'euthyna-coverage', + format: 'lcov', + count: facts.length + }); + } + + return { facts, evaluated, notEvaluated, measured }; +} + /** * True for coverage.py's `{"meta": {...}, "files": {...}}` shape. * A c8 report never has a top-level `meta` key. @@ -273,6 +734,7 @@ function isCoveragePyReport(coverage) { */ export function detectCoverageFormat(coverage) { if (coverage && typeof coverage === 'object' && coverage.__go) return 'go'; + if (coverage && typeof coverage === 'object' && coverage.__lcov) return 'lcov'; if (isCoveragePyReport(coverage)) return 'coverage.py'; const entries = Object.entries(coverage).filter(([, v]) => v && typeof v === 'object'); @@ -405,6 +867,12 @@ export async function collectCoverageFacts({ coverageFile, targets = [], source, return collectGoCoverageFacts({ coverage, coverageFile, targets, source, lang }); } + // lcov tracefiles carry mangled function names, so symbol location works on + // the function table and needs no source tree — the opposite trade from Go. + if (coverage.__lcov) { + return collectLcovCoverageFacts({ coverage, coverageFile, targets, lang }); + } + // c8: entries are the report's file keys directly. // coverage.py: the file entries live under `files`. const entries = Object.entries(isPy ? (coverage.files ?? {}) : coverage).filter( diff --git a/test/coverage-lcov.test.js b/test/coverage-lcov.test.js new file mode 100644 index 0000000..7993f94 --- /dev/null +++ b/test/coverage-lcov.test.js @@ -0,0 +1,620 @@ +/** + * lcov tracefile support (`llvm-cov export --format=lcov`, gcov `.info`, + * cargo-llvm-cov) — the Rust gap recorded in docs/case-study-rust-zh.md §7. + * + * The tracefile is text and *does* carry function names, so unlike a Go profile + * this path needs no source tree. The price is that the names are mangled, so + * the matching runs on the mangled form. The behaviour worth defending is the + * same as in the other coverage suites — what the producer refuses to say — + * plus the lcov-specific refusals: a function with no count at all, a file + * absent from the tracefile, a file with no function table, and a record whose + * body does not parse. + * + * The fixture shape is copied from a real tracefile: a probe crate built with + * `RUSTFLAGS=-C instrument-coverage`, run under `cargo test`, merged with + * `llvm-profdata merge -sparse`, exported with `llvm-cov export --format=lcov`. + * The mangled names below are that run's output, not invented strings. + */ +import { test, describe, before, after } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import path from 'node:path'; + +import { + collectCoverageFacts, + loadCoverage, + detectCoverageFormat, + parseLcov, + lcovSymbolMatches, + lcovSymbolNeedle +} from '../src/facts/coverage.js'; +import { STATUS } from '../src/contract.js'; + +let dir; + +before(async () => { + dir = await mkdtemp(path.join(tmpdir(), 'euthyna-cov-lcov-')); +}); + +after(async () => { + if (dir) await rm(dir, { recursive: true, force: true }); +}); + +const CRATE_HASH = 'CsVynAXQWAN6'; + +/** Write a tracefile fixture and hand back its path. */ +async function writeLcov(name, text) { + const file = path.join(dir, name); + await writeFile(file, text, 'utf8'); + return file; +} + +const LIB = () => path.join(dir, 'src', 'lib.rs'); +const OTHER = () => path.join(dir, 'src', 'other.rs'); + +/** The single function entry for `symbol` in `file`, from a parsed tracefile. */ +function fnEntry(parsed, file, symbol) { + return parsed.files.get(file)?.functions.get(symbol); +} + +/** + * One record for a module with three same-or-differently-qualified names: + * top-level `dup`, `nested::dup`, and a never-invoked `never_called`. + */ +function tracefile() { + return [ + `SF:${LIB()}`, + `FN:3,_RNv${CRATE_HASH}_10lcov_probe7long_fn`, + `FN:28,_RNv${CRATE_HASH}_10lcov_probe3dup`, + `FN:23,_RNvNt${CRATE_HASH}_10lcov_probe6nested3dup`, + `FN:11,_RNv${CRATE_HASH}_10lcov_probe12never_called`, + `FN:48,_RINv${CRATE_HASH}_10lcov_probe10generic_fnReEB2_`, + `FN:48,_RINv${CRATE_HASH}_10lcov_probe10generic_fnlEB2_`, + `FN:64,_RNvXs_${CRATE_HASH}_10lcov_probeNtB4_4GearNtB4_6Widget5label`, + `FNDA:1,_RNv${CRATE_HASH}_10lcov_probe7long_fn`, + `FNDA:1,_RNv${CRATE_HASH}_10lcov_probe3dup`, + `FNDA:1,_RNvNt${CRATE_HASH}_10lcov_probe6nested3dup`, + `FNDA:0,_RNv${CRATE_HASH}_10lcov_probe12never_called`, + `FNDA:1,_RINv${CRATE_HASH}_10lcov_probe10generic_fnReEB2_`, + `FNDA:1,_RINv${CRATE_HASH}_10lcov_probe10generic_fnlEB2_`, + `FNDA:1,_RNvXs_${CRATE_HASH}_10lcov_probeNtB4_4GearNtB4_6Widget5label`, + 'FNF:7', + 'FNH:6', + 'DA:3,1', + 'DA:11,0', + 'DA:28,1', + 'LF:3', + 'LH:2', + 'BRF:0', + 'BRH:0', + 'end_of_record', + `SF:${OTHER()}`, + `FN:1,_RNvNt${CRATE_HASH}_10lcov_probe5other13only_in_other`, + 'FNDA:0,_RNvNt' + CRATE_HASH + '_10lcov_probe5other13only_in_other', + 'FNF:1', + 'FNH:0', + 'DA:1,0', + 'LF:1', + 'LH:0', + 'end_of_record' + ].join('\n'); +} + +describe('lcov tracefiles', () => { + test('the loader identifies a tracefile by shape and names the format', async () => { + const file = await writeLcov('basic.info', tracefile()); + const coverage = await loadCoverage(file); + assert.equal(coverage.__lcov, true); + assert.equal(coverage.files.size, 2); + assert.equal(detectCoverageFormat(coverage), 'lcov'); + }); + + test('an invoked function yields unknown, never "executed"', async () => { + const file = await writeLcov('invoked.info', tracefile()); + const { facts } = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'long_fn', file: LIB() }] + }); + assert.equal(facts.length, 1); + assert.equal(facts[0].status, STATUS.UNKNOWN); + assert.equal(facts[0].detail.reason, 'nonzero_count_cannot_prove_call_site_execution'); + assert.equal(facts[0].evidence.line, 3); + assert.equal(facts[0].method, 'command'); + // The whole point of the producer: nothing here says a call site ran. + assert.doesNotMatch(JSON.stringify(facts[0]), /"status":\s*"executed"/i); + }); + + test('a zero count is established as never invoked, with the mangled name on record', async () => { + const file = await writeLcov('zero.info', tracefile()); + const { facts, evaluated, measured } = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'never_called', file: LIB() }] + }); + assert.equal(measured, true); + assert.equal(facts.length, 1); + assert.equal(facts[0].status, STATUS.ESTABLISHED); + assert.equal(facts[0].detail.reason, 'invocation_count_zero'); + assert.equal(facts[0].detail.invocationCount, 0); + assert.equal(facts[0].evidence.line, 11); + assert.equal(facts[0].detail.mangled, `_RNv${CRATE_HASH}_10lcov_probe12never_called`); + assert.equal(evaluated[0].format, 'lcov'); + }); + + test('a bare name matching two symbols reports both, each naming its own symbol', async () => { + const file = await writeLcov('bare.info', tracefile()); + const { facts } = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'dup', file: LIB() }] + }); + assert.equal(facts.length, 2); + const mangled = facts.map(f => f.detail.mangled).sort(); + assert.deepEqual(mangled, [ + `_RNv${CRATE_HASH}_10lcov_probe3dup`, + `_RNvNt${CRATE_HASH}_10lcov_probe6nested3dup` + ].sort()); + for (const fact of facts) { + // Each fact names which of the two it is about — never a claim about "dup" at large. + assert.match(fact.statement, /本次查询匹配到 2 个符号/); + assert.ok(fact.statement.includes(fact.detail.mangled)); + } + }); + + test('a module-qualified query answers for exactly one of two same-named functions', async () => { + const file = await writeLcov('qualified.info', tracefile()); + const { facts } = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'nested::dup', file: LIB() }] + }); + assert.equal(facts.length, 1); + assert.equal(facts[0].detail.mangled, `_RNvNt${CRATE_HASH}_10lcov_probe6nested3dup`); + assert.equal(facts[0].evidence.line, 23); + }); + + test('one generic function instantiated twice reports one fact per instantiation', async () => { + const file = await writeLcov('generic.info', tracefile()); + const { facts } = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'generic_fn', file: LIB() }] + }); + assert.equal(facts.length, 2); + for (const fact of facts) assert.equal(fact.evidence.line, 48); + assert.deepEqual( + facts.map(f => f.detail.mangled).sort(), + [ + `_RINv${CRATE_HASH}_10lcov_probe10generic_fnReEB2_`, + `_RINv${CRATE_HASH}_10lcov_probe10generic_fnlEB2_` + ] + ); + }); + + test('a trait impl answers both for the type-qualified and the bare method name', async () => { + const file = await writeLcov('trait.info', tracefile()); + const byType = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'Gear::label', file: LIB() }] + }); + const bare = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'label', file: LIB() }] + }); + assert.equal(byType.facts.length, 1); + assert.equal(bare.facts.length, 1); + assert.equal(byType.facts[0].detail.mangled, bare.facts[0].detail.mangled); + }); + + test('an already-mangled query is matched exactly and never half-decoded', async () => { + const file = await writeLcov('mangled.info', tracefile()); + const exact = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: `_RNv${CRATE_HASH}_10lcov_probe7long_fn`, file: LIB() }] + }); + assert.equal(exact.facts.length, 1); + assert.equal(exact.facts[0].detail.invocationCount, 1); + + const partial = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: `_RNv${CRATE_HASH}_10lcov_probe7long`, file: LIB() }] + }); + assert.equal(partial.facts.length, 1); + assert.equal(partial.facts[0].detail.reason, 'symbol_not_located_in_fnmap'); + }); + + test('a function with an FN line and no FNDA line is not evaluated, never "never invoked"', async () => { + const file = await writeLcov( + 'nofnda.info', + [`SF:${LIB()}`, `FN:9,_RNv${CRATE_HASH}_10lcov_probe5ghost`, 'FNF:1', 'FNH:0', 'end_of_record'].join('\n') + ); + const { facts, notEvaluated, evaluated, measured } = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'ghost', file: LIB() }] + }); + assert.equal(facts.length, 0); + assert.equal(measured, false); + assert.equal(evaluated.length, 0); + assert.equal(notEvaluated.length, 1); + assert.match(notEvaluated[0].reason, /缺失的计数不是 0/); + }); + + test('an FNDA with no FN line keeps its count and leaves the line unknown', async () => { + const parsed = parseLcov( + [`SF:${LIB()}`, `FNDA:0,_RNv${CRATE_HASH}_10lcov_probe7orphan`, 'end_of_record'].join('\n') + ); + assert.equal(parsed.files.size, 1); + const orphan = parsed.files.get(LIB()).functions.get(`_RNv${CRATE_HASH}_10lcov_probe7orphan`); + // The count is in the file, so it is kept; only the line is unknown. + assert.deepEqual(orphan, { line: null, count: 0 }); + }); + + test('two records for one file merge by summing, never by overwriting', async () => { + const symbol = `_RNv${CRATE_HASH}_10lcov_probe7split`; + const zeroThenTwo = parseLcov( + [ + `SF:${LIB()}`, + `FN:1,${symbol}`, + `FNDA:0,${symbol}`, + 'end_of_record', + `SF:${LIB()}`, + `FN:1,${symbol}`, + `FNDA:2,${symbol}`, + 'end_of_record' + ].join('\n') + ); + // Summing is the one merge that cannot turn a nonzero count into zero, and + // zero is the only conclusion this producer draws. + assert.equal(fnEntry(zeroThenTwo, LIB(), symbol).count, 2); + + const twoThenZero = parseLcov( + [ + `SF:${LIB()}`, + `FN:1,${symbol}`, + `FNDA:2,${symbol}`, + 'end_of_record', + `SF:${LIB()}`, + `FN:1,${symbol}`, + `FNDA:0,${symbol}`, + 'end_of_record' + ].join('\n') + ); + assert.equal(fnEntry(twoThenZero, LIB(), symbol).count, 2); + }); + + test('a missing count stays missing when records merge — null is not a zero', () => { + const symbol = `_RNv${CRATE_HASH}_10lcov_probe7maybe`; + const parsed = parseLcov( + [ + `SF:${LIB()}`, + `FN:1,${symbol}`, + `FNDA:4,${symbol}`, + 'end_of_record', + `SF:${LIB()}`, + `FN:1,${symbol}`, + 'end_of_record' // no FNDA at all + ].join('\n') + ); + // 4 + null must not collapse to 4 (a count from one record is not a count for + // the file) and must not become 0 — 0 is the only number this producer draws + // a conclusion from, so a wrong zero here would fabricate a real finding. + assert.equal(fnEntry(parsed, LIB(), symbol).count, null); + }); + + test('a duplicate FN line keeps the first line rather than the last', () => { + const symbol = `_RNv${CRATE_HASH}_10lcov_probe7twice`; + const parsed = parseLcov( + [`SF:${LIB()}`, `FN:1,${symbol}`, `FN:9,${symbol}`, `FNDA:0,${symbol}`, 'end_of_record'].join('\n') + ); + // The line ends up in the fact's evidence, so which one wins is not cosmetic. + assert.equal(fnEntry(parsed, LIB(), symbol).line, 1); + }); + + test('two FNDA lines for one symbol sum rather than the last one winning', () => { + const symbol = `_RNv${CRATE_HASH}_10lcov_probe7twice`; + const parsed = parseLcov( + [`SF:${LIB()}`, `FN:1,${symbol}`, `FNDA:3,${symbol}`, `FNDA:4,${symbol}`, 'end_of_record'].join('\n') + ); + assert.equal(fnEntry(parsed, LIB(), symbol).count, 7); + }); + + test('a file absent from the tracefile is not evaluated, not "never executed"', async () => { + const file = await writeLcov('absent.info', tracefile()); + const missing = path.join(dir, 'src', 'not_instrumented.rs'); + const { facts, notEvaluated, measured } = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'anything', file: missing }] + }); + assert.equal(facts.length, 0); + assert.equal(measured, false); + assert.equal(notEvaluated.length, 1); + assert.match(notEvaluated[0].reason, /「未出现」不能当「未执行」/); + // Deliberately the opposite of the c8 path, where absence is established. A + // notEvaluated entry is not a fact and carries no status field at all, so this + // asserts that absence — "not established" would hold even for a malformed + // entry, because the field does not exist to be compared. + assert.equal('status' in notEvaluated[0], false); + }); + + test('a symbol that is not in the table is unknown, and says why a mangled name may not match', async () => { + const file = await writeLcov('unlocated.info', tracefile()); + const { facts, measured } = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'no_such_fn', file: LIB() }] + }); + assert.equal(measured, true); + assert.equal(facts.length, 1); + assert.equal(facts[0].status, STATUS.UNKNOWN); + assert.equal(facts[0].detail.reason, 'symbol_not_located_in_fnmap'); + assert.equal(facts[0].method, 'static'); + assert.match(facts[0].statement, /lcov 的 FN: 名是 mangled 的/); + // The needle it tried is in the statement, so a reader can see what was looked for. + assert.ok(facts[0].statement.includes('10no_such_fn')); + }); + + test('a tracefile with no function table is not evaluated — line data cannot answer a symbol question', async () => { + const file = await writeLcov( + 'linesonly.info', + [`SF:${LIB()}`, 'DA:1,1', 'DA:2,0', 'LF:2', 'LH:1', 'end_of_record'].join('\n') + ); + const { facts, notEvaluated, measured } = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'anything', file: LIB() }] + }); + assert.equal(facts.length, 0); + assert.equal(measured, false); + assert.equal(notEvaluated.length, 1); + assert.match(notEvaluated[0].reason, /没有任何 FN: 行/); + }); + + test('a query answered only by refusals is measured: false, never clean', async () => { + const file = await writeLcov('refusals.info', tracefile()); + const { facts, evaluated, measured } = await collectCoverageFacts({ + coverageFile: file, + targets: [ + { symbol: 'anything', file: path.join(dir, 'src', 'absent_a.rs') }, + { symbol: 'anything', file: path.join(dir, 'src', 'absent_b.rs') } + ] + }); + assert.equal(facts.length, 0); + assert.equal(measured, false); + assert.equal(evaluated.length, 0, 'a query that produced no fact was not answered'); + }); + + test('a bare name with no file restriction still searches every record', async () => { + const file = await writeLcov('unrestricted.info', tracefile()); + const { facts } = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'only_in_other' }] + }); + assert.equal(facts.length, 1); + assert.equal(facts[0].status, STATUS.ESTABLISHED); + assert.equal(facts[0].evidence.file, OTHER()); + }); + + test('comments and summary lines are recognised and ignored, not treated as unknown keys', async () => { + const parsed = parseLcov( + [ + 'TN:', + `SF:${LIB()}`, + `FN:5,_RNv${CRATE_HASH}_10lcov_probe5hello`, + `FNDA:1,_RNv${CRATE_HASH}_10lcov_probe5hello`, + 'FNF:1', + 'FNH:1', + 'BRDA:5,0,0,1', + 'LF:1', + 'LH:1', + 'end_of_record' + ].join('\n') + ); + assert.equal(parsed.files.size, 1); + assert.deepEqual( + fnEntry(parsed, LIB(), `_RNv${CRATE_HASH}_10lcov_probe5hello`), + { line: 5, count: 1 } + ); + }); + + test('a malformed body is refused with a reason, not interpreted', async () => { + for (const [name, text] of [ + ['unclosed.info', `SF:${LIB()}\nFN:1,_RNa\n`], + ['fnoutofrec.info', `TN:\nFN:1,_RNa\nSF:${LIB()}\nend_of_record\n`], + ['fnbadvalue.info', `SF:${LIB()}\nFN:notanumber,_RNa\nend_of_record\n`], + ['nosf.info', 'FNF:0\nend_of_record\n'] + ]) { + const file = await writeLcov(name, text); + const { facts, notEvaluated, measured } = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'anything', file: LIB() }] + }); + assert.equal(facts.length, 0, name); + assert.equal(measured, false, name); + assert.equal(notEvaluated.length, 1, name); + assert.match(notEvaluated[0].reason, /无法读取覆盖率数据|cannot read coverage data/, name); + } + }); + + test('the newer FNL/FNA encoding is refused rather than read out of a guessed shape', async () => { + for (const key of ['FNL', 'FNA', 'FNAB']) { + const file = await writeLcov( + `newenc-${key}.info`, + [`SF:${LIB()}`, `${key}:1,2,_RNv${CRATE_HASH}_10lcov_probe5hello`, 'end_of_record'].join('\n') + ); + const { facts, notEvaluated, measured } = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'hello', file: LIB() }] + }); + assert.equal(facts.length, 0, key); + assert.equal(measured, false, key); + assert.match(notEvaluated[0].reason, /无法读取覆盖率数据/, key); + } + }); + + test('the parser refuses each unaccounted-for line instead of skipping it', () => { + const bad = [ + `SF:${LIB()}\nFN:1,_RNa\n`, // unclosed + `FN:1,_RNa\nSF:${LIB()}\nend_of_record\n`, // function before any record + `SF:${LIB()}\nFN:nope,_RNa\nend_of_record\n`, // FN value is not "," + `SF:${LIB()}\n FN:1,_RNa\nend_of_record\n`, // indented: not an lcov line + `SF:${LIB()}\nFNL:1,2,_RNa\nend_of_record\n`, // unsupported encoding + 'TN:\nend_of_record\n', // no SF record at all + `SF:${LIB()}\nFN:1,_RNa\nSF:${OTHER()}\nend_of_record\n` // second SF inside a record + ]; + for (const text of bad) assert.equal(parseLcov(text), null, JSON.stringify(text)); + }); + + test('a closed record followed by an unclosed one is refused, not half-read', () => { + // The lone-unclosed case above is caught by the "no SF: record at all" guard, + // so it does not exercise the EOF guard: with one *complete* record already in + // hand, dropping the trailing record would look like success. Measured against + // the shipped parser — refusing is what keeps the second record's functions + // out of the answer instead of silently absent from it. + const text = [ + `SF:${LIB()}`, + `FN:1,_RNv${CRATE_HASH}_10lcov_probe5first`, + `FNDA:1,_RNv${CRATE_HASH}_10lcov_probe5first`, + 'end_of_record', + `SF:${OTHER()}`, + `FN:2,_RNv${CRATE_HASH}_10lcov_probe6second`, + `FNDA:0,_RNv${CRATE_HASH}_10lcov_probe6second` + // no end_of_record + ].join('\n'); + assert.equal(parseLcov(text), null); + }); + + test('a symbol query needs no --source: the tracefile carries the names', async () => { + const file = await writeLcov('nosource.info', tracefile()); + const withSource = await collectCoverageFacts({ + coverageFile: file, + source: dir, + targets: [{ symbol: 'never_called', file: LIB() }] + }); + const without = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'never_called', file: LIB() }] + }); + assert.equal(withSource.facts.length, 1); + assert.equal(without.facts.length, 1); + assert.deepEqual(without.facts[0].status, withSource.facts[0].status); + }); + + test('a missing file is not evaluated, never "not covered"', async () => { + const { facts, notEvaluated, measured } = await collectCoverageFacts({ + coverageFile: path.join(dir, 'nope.info'), + targets: [{ symbol: 'x', file: LIB() }] + }); + assert.equal(facts.length, 0); + assert.equal(measured, false); + assert.equal(notEvaluated.length, 1); + }); + + test('a path query states the symbol it resolved to, not the pattern', async () => { + const file = await writeLcov( + 'subject.info', + [ + `SF:${LIB()}`, + `FN:28,_RNv${CRATE_HASH}_10lcov_probe3dup`, + `FN:23,_RNvNt${CRATE_HASH}_10lcov_probe6nested3dup`, + `FNDA:1,_RNv${CRATE_HASH}_10lcov_probe3dup`, + `FNDA:0,_RNvNt${CRATE_HASH}_10lcov_probe6nested3dup`, + 'FNF:2', + 'FNH:1', + 'end_of_record' + ].join('\n') + ); + const { facts } = await collectCoverageFacts({ + coverageFile: file, + targets: [{ symbol: 'lcov_probe::nested::dup', file: LIB() }] + }); + // A `::` query is a pattern, so the statement names the symbol that matched: + // "Symbol lcov_probe::nested::dup was never invoked" would claim something + // about a name the tracefile does not carry. + assert.equal(facts.length, 1); + assert.match(facts[0].statement, /_RNvNt/); + assert.doesNotMatch(facts[0].statement, /lcov_probe::nested::dup/); + assert.equal(facts[0].detail.mangled, `_RNvNt${CRATE_HASH}_10lcov_probe6nested3dup`); + }); +}); + +describe('lcov symbol matching', () => { + const name = mangled => `_RNvNt${CRATE_HASH}_10lcov_probe6${mangled}`; + + test('a segment is looked up as ', () => { + assert.equal(lcovSymbolNeedle(['dup']), '3dup'); + assert.equal(lcovSymbolNeedle(['nested', 'dup']), '6nested3dup'); + // Lengths are bytes, not code points: a non-ASCII identifier still works. + assert.equal(lcovSymbolNeedle(['名']), '3名'); + }); + + test('segments are anchored at the start, after a mangling separator, or at a proven pair boundary', () => { + assert.equal(lcovSymbolMatches(['dup'], `${name('nested3dup')}`), true); + assert.equal(lcovSymbolMatches(['dup'], `_RNv${CRATE_HASH}_10lcov_probe7xyz3dup`), false); + assert.equal(lcovSymbolMatches(['bump'], `_RNvMCsVynAXQWAN6_10lcov_probeNtB2_7Counter4bump`), true); + }); + + test('segments match independently, so a disambiguator between them is tolerated', () => { + // A #[cfg(test)] module comes out as `testss_`, not `tests`; a concatenated + // needle would miss this outright. + assert.equal( + lcovSymbolMatches(['tests', 'exercises_some'], `_RNvNt${CRATE_HASH}_10lcov_probe5testss_14exercises_some`), + true + ); + }); + + test('a qualified query does not answer for the same name in another module', () => { + assert.equal(lcovSymbolMatches(['nested', 'dup'], `_RNv${CRATE_HASH}_10lcov_probe3dup`), false); + }); + + test('an already-mangled query is matched exactly', () => { + const mangled = `_RNv${CRATE_HASH}_10lcov_probe7long_fn`; + assert.equal(lcovSymbolMatches([mangled], mangled), true); + assert.equal(lcovSymbolMatches([mangled.slice(0, -3)], mangled), false); + }); + + test('an empty segment list matches nothing but does not throw', () => { + assert.equal(lcovSymbolMatches([], `_RNv${CRATE_HASH}_10lcov_probe3dup`), false); + }); + + test('a query never answers for an identifier that sits inside a longer one', () => { + // Measured on a real `llvm-cov export` tracefile: `get` and `cfg_3get` are two + // functions, and `8cfg_3get` is itself a complete `` pair, so + // `3get` sits inside an identifier rather than at a boundary. Before the + // containment guard a query for `get` reported "Symbol get was not invoked + // even once" about `cfg_3get` — an established fact about a function nobody + // asked about, and false about `get`, which had been called once. + const get = '_RNvCsi8SMnIpCoEi_13collide_probe3get'; + const cfg = '_RNvCsi8SMnIpCoEi_13collide_probe8cfg_3get'; + assert.equal(lcovSymbolMatches(['get'], get), true); + assert.equal(lcovSymbolMatches(['get'], cfg), false); + assert.equal(lcovSymbolMatches(['collide_probe', 'get'], get), true); + assert.equal(lcovSymbolMatches(['collide_probe', 'get'], cfg), false); + // The collision name is still reachable on its own — the guard rejects a + // needle that lands inside a longer identifier, not the identifier itself. + assert.equal(lcovSymbolMatches(['cfg_3get'], cfg), true); + }); + + test('each repeated segment consumes the name it matched', () => { + // The cursor advances past every match, so one identifier cannot satisfy two + // segments of the same query. + assert.equal(lcovSymbolMatches(['a', 'a'], '1a'), false); + assert.equal(lcovSymbolMatches(['a', 'a'], '_RNv__1a1a'), true); + }); + + test('a qualified query may skip components between its segments', () => { + // A trait method's path carries its Self type — `Gear::Widget::label` — so + // `Gear::label` names a subsequence. Measured: the contiguous form of this + // matcher rejected it against the real tracefiles, which is why the segments + // are matched independently rather than as one concatenated needle. + const label = `_RNvXs_${CRATE_HASH}_10lcov_probeNtB4_4GearNtB4_6Widget5label`; + assert.equal(lcovSymbolMatches(['Gear', 'label'], label), true); + assert.equal(lcovSymbolMatches(['Widget', 'label'], label), true); + assert.equal(lcovSymbolMatches(['Gear', 'Widget', 'label'], label), true); + }); + + test('a qualified query is a subsequence test, not a path-exact one', () => { + // Recorded rather than hidden: `lcov_probe::dup` also answers for + // `lcov_probe::nested::dup`. Each fact names the symbol it is about, so a + // reader can tell the two apart; narrowing this further would need the + // mangling's grammar. + const outer = `_RNv${CRATE_HASH}_10lcov_probe3dup`; + const nested = `_RNvNt${CRATE_HASH}_10lcov_probe6nested3dup`; + assert.equal(lcovSymbolMatches(['lcov_probe', 'dup'], outer), true); + assert.equal(lcovSymbolMatches(['lcov_probe', 'dup'], nested), true); + }); +});