Skip to content

feat(tools): add code coverage tools #104

Closed

Description

@CalvinAllen

Part of #95. Depends on #101 for run-completion tracking.

Add code coverage tools — trigger an analysis run and read the results back.

Proposed tools

Tool Behavior
coverage_analyze Run code coverage analysis over all tests
coverage_results Return coverage numbers from the last run
coverage_show Open the Code Coverage Results window

Triggering a run

DTE.ExecuteCommand("Test.AnalyzeCodeCoverageForAllTests"), and Test.CodeCoverageResults to surface the window.

Reuse the IOperationState completion tracking from #101 — VS 2026 raises an AnalyzeCodeCoverageOperationStateChanged signal, and the run needs to finish before results are readable.

Reading results — two routes

Route A — VS 2026 brokered service (native, but limited)

Microsoft.CodeCoverage.VisualStudio.Contracts.dll:

Task<CoverageRunData> IReportService.GetReportAsync(string inputPath, CancellationToken ct);
Task<string[]>        IReportService.MergeCoverageFilesAsync(string outputPath, string[] files, MergeOperation op, CancellationToken ct);

class CoverageFileData { string Path; int LinesCovered; int LinesNotCovered; int LinesPartiallyCovered; LineCoverageData[] LinesCoverageData; }
class LineCoverageData { LineCoverageStatus CoverageStatus; int StartLine; int StartColumn; int EndLine; int EndColumn; int BlocksCovered; }
enum  LineCoverageStatus { Covered, NotCovered, PartiallyCovered }

Drawbacks: VS 2026 only (the assembly doesn't exist in VS 2022), granularity is per-file and per-line only — no class or module rollup — and unlike the TestWindow and Terminal assemblies it has no binding redirect or codeBase in devenv.exe.config, so resolution needs handling ourselves.

Route B — dotnet-coverage / Microsoft.CodeCoverage.Console (portable, recommended)

Convert .coverage → Cobertura and parse. Works on both VS versions, no dependency on unsupported internal assemblies, and gives a proper module → class → method hierarchy, which is what was actually asked for in #95.

Leaning toward B as the primary implementation, with A as a possible optimization on VS 2026. Open to input.

Edition gating

Code coverage was Enterprise-only through VS 2022; it became available in Community and Professional in VS 2026. Verified on disk — CommonExtensions\Microsoft\AnalyzeCodeCoverage\ exists in VS 2026 Community but not VS 2022 Community.

The tools must degrade gracefully: detect availability and return a clear "code coverage is not available in this edition of Visual Studio" message rather than failing opaquely or hanging on a disabled command. DTE.ExecuteCommand throws when the command is disabled, so that needs catching explicitly.

Out of scope

A class/module coverage tree read directly from the VS API — that granularity doesn't exist in the contracts. Route B covers it instead.

Activity

  1. CalvinAllen commented on Sep 3, 2026

    @CalvinAllen
    ContributorAuthor

    Correction to my own note above, found while implementing #101/#102 (merged in #105).

    I framed the assembly problem as specific to Microsoft.CodeCoverage.VisualStudio.Contracts.dll — "unlike the TestWindow and Terminal assemblies it has no binding redirect or codeBase in devenv.exe.config, so resolution needs handling ourselves." That implied the other assemblies were straightforward to reference. They aren't, and the reason has nothing to do with redirects:

    $ dotnet msbuild ...MCPServer.csproj -getProperty:DevEnvDir -getProperty:VsInstallRoot
    { "DevEnvDir": "*Undefined*", "VsInstallRoot": "" }
    

    $(DevEnvDir) is only defined under a Visual Studio / developer command prompt build. This repo uses dotnet build locally and in CI, so no VS-install-directory assembly can be referenced by HintPath, redirect or not. Binding redirects and codeBase entries only affect runtime resolution.

    This makes Route B the clear choice, not just the leaning one. Shelling out to dotnet-coverage / Microsoft.CodeCoverage.Console and parsing Cobertura:

    • has no build-time dependency on anything in the VS install
    • works on both VS 2022 and VS 2026
    • gives the module → class → method hierarchy feat(tools): Run tests #95 actually asked for, which IReportService cannot (it is per-file and per-line only)

    Route A would need the same reflection treatment as Services/TestExplorerInterop.cs and would still only deliver file-level granularity on VS 2026 alone. Hard to justify against Route B.

    Unchanged: the edition gating (Enterprise-only through VS 2022, all editions in VS 2026) still needs graceful degradation, and DTE.ExecuteCommand still throws on a disabled command — #105 added an IsCommandAvailable probe in VisualStudioService that can be reused for Test.AnalyzeCodeCoverageForAllTests.

  2. self-assigned this
    on Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions