ci: say which test failed, on the run's front page - #137
Merged
Conversation
A red `build-test` named the failing test only in the raw job log — a scroll through ~2000 test lines to find one. Worse, `build/coverage.sh` aborted on the failing `dotnet test` under `set -e`, so no report was rendered, the coverage summary printed nothing and the artifact upload failed for a missing Cobertura.xml: two extra red steps burying the one that mattered. - `build/test-summary.py` parses the trx and prints a verdict line, a per-assembly table and a `<details>` block per failure (message in full, stack trimmed to six frames). `--annotate` also emits `::error::` commands, which puts each failure on the Files-changed tab. - `build/coverage.sh` asks for a trx logger, carries the test exit status to the end instead of aborting, and tolerates reportgenerator finding nothing. - CI publishes both summaries with `if: always()`, and the examples run now writes a trx too. The trx is parsed rather than the console output because the console format is a rendering choice of whichever logger was enabled; the trx is the record.
VDBBjorn
enabled auto-merge (squash)
August 4, 2026 19:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #139
What changed
A red
build-testused to name the failing test only in the raw job log — on ~2000 teststhat is a scroll to find one line, and the run's front page said nothing at all.
It was worse than that.
build/coverage.shruns underset -e, so the failingdotnet testaborted the script: no Cobertura.xml was rendered, the coverage summary printed nothing, and
Upload coveragefailed onif-no-files-found: error. Two extra red steps, both burying theone that mattered.
Three changes:
build/test-summary.py(new) parses.trxand prints a verdict line, a per-assemblytable, and a
<details>block per failure — message in full, stack trimmed to six frames.--annotateadditionally emits::error title=…::workflow commands, which is what puts afailure on the Files-changed tab next to the test that produced it. It parses the trx rather
than console output because the console format is a rendering choice of whichever logger was
enabled, while the trx is the recorded result. Finding no trx files is not an error: a job
that died before any test ran has nothing to summarise, and a stack trace from this script
on top of the real failure helps nobody.
build/coverage.shasks for a trx logger, carries thedotnet testexit status to theend instead of aborting on it, and tolerates reportgenerator finding nothing to report. It
still exits non-zero, so the step is still red — it just stops taking the rest of the job
down with it.
.github/workflows/ci.ymlpublishes both summaries to$GITHUB_STEP_SUMMARYwithif: always()(the patternmutation.ymlalready uses), and the examples run now writes atrx of its own so its failures land there too.
Verified locally both ways: a passing run renders
with the per-assembly table, and a deliberately failing test renders 🔴 failed, the
<details>block with xUnit's full message and a six-frame stack, and the matching::error title=…::line.Checklist
dotnet testpasses locally (Postgres-backed tests need a running Docker daemon)PublicAPI.Unshipped.txt/PublicAPI.Shipped.txt— no public API touchedbreaking-changelabel applied if a consumer breaks — nothing breaks; build scripts and workflow onlyCHANGELOG.mdnot hand-edited; the release workflow drafts it