Skip to content

Latest commit

 

History

History
163 lines (124 loc) · 7.89 KB

coverage.md

File metadata and controls

163 lines (124 loc) · 7.89 KB

Coverage: what is measured, what is not, and what stands in for it

Eclipse ThreadX ZoneX. Last updated 3 September 2026.

ZoneX does not hold the Eclipse ThreadX suite's usual coverage threshold over its whole repository, and docs/decisions.md D11 records why. This document is the other half of that decision: which files are held to a floor, what the floor is, which files are not, and what the evidence for those is instead.

A coverage number quoted without that division is worth very little. A reviewer reading "72% of ZoneX" learns nothing about whether the parts that can be tested were.


The floor: 100% of the fully reachable core, lines and branches

test/host/run.sh coverage enforces it, and CI runs that script on every push and pull request. It is not a report; it fails the build.

file what it is
core/src/zx_fault.c the fault classifier and syndrome decoder
core/src/zx_fault_log.c the many-fault log
core/src/zx_guest_console.c the guest console's tagging rules
core/src/zx_manifest_verify.c the manifest validator
core/src/zx_mm_setup.c the region-layout planner
core/src/zx_partition_manager.c the partition loader
core/src/zx_schedule.c the time-partition schedule

100% is not aspirational — it is what they measure. All seven are pure functions with no hardware in them, all seven are reachable in full from a workstation, and none has an excuse for an unreached line. A validator rule added without a case that fails it drops the number and fails the build, which is the entire point: a rule nothing has ever seen reject anything is not a rule, it is a comment.

Branches as well as lines, and that is the half that does the work. Line coverage is satisfied by a run that takes one arm of every if. The failures these files exist to prevent live in the arms nobody took — the manifest that is refused, the fault that is a read rather than a write, the partial line closed before the hypervisor speaks.

A file joins this list when it becomes reachable in full

Not when it is written. Two files joined it during the isolation work — the fault decoder and the fault log — once the isolation matrix made them reachable: the decoder's "on a READ" arm had never been printed, because every fault the suite provoked happened to be a write, and the matrix provokes a read and a write against the same address deliberately.

Adding a file that cannot be reached in full is how a floor ends up lowered to whatever passes. The list is short on purpose.


What is not measured by gcov, and what covers it instead

platform/cortex_r52/ — the port

CP15 assembly, EL2 vector tables, MPU region programming, GIC and generic timer setup, and the context switch. gcov cannot reach any of it from a workstation, and instrumenting it on the target would change the code generation of the thing being measured — which for a file whose subject is a partition switch's cost is not a small objection.

The evidence is the model and the board. Specifically:

what covered by
stage-2 region programming, and that limits are EXACT the isolation matrix, and the widened build that proves the adjacent-granule check can fail
the trap vector, all four exception classes the probe image: a violation (EC 0x24), a hypercall (EC 0x12), an unexpected class (EC 0x03, via HCR.TID1), and ZoneX faulting on itself (EC 0x25, in its own build)
the window boundary and the context switch the two-partition image and the determinism regression, on both targets
the EL2 MPU enable order, and the refusal when regions are short the starved build, which refuses to start and says why
the time freeze the no_freeze build, which must fail and does
the boundary's own tick the no_tick build, which must refuse rather than hang
halt-on-fault, the shipping policy the halt build, run on both targets
the unresumable vectors' stack reset and depth guard the refault build, which makes the fault report fault on its own console write and is judged from memory because it has none

Every one of those is a build that must FAIL, or a build that must pass with a named check. That is the substitute for structural coverage here, and it is a weaker guarantee in one direction and a stronger one in another: it says nothing about unreached lines, and it says a great deal about whether the reached ones do what they claim.

examples/ — the demonstration images

Not measured and not a target. They are the test harness, not the product.

Structural coverage of target code belongs to the certification phase

MC/DC and structural coverage of the port belong to the certification phase and need their own tooling — an instrumenting target build, or a trace-based measurement off the ETM. They are not attempted here and no number in this repository should be read as standing in for them.


Configurations, and the trap of merging them

gcovr merges across the host suite's translation units, and the floor is enforced on that merged report. But the interesting question is not units, it is configurations: code that one build compiles and another does not.

ZoneX has three such axes, and every arm of all three is compiled somewhere in the matrix:

condition one arm the other arm
ZX_RESET_IN_THUMB the S32Z280 build, where the core resets in T32 and needs a trampoline the FVP build, which resets in A32
ZX_CONSOLE_BOARD the S32Z280 build, a polled LINFlexD the FVP build, semihosting
ZX_GUEST_B_BLOB every two-partition image every one-partition image

This is the trap the Cortex-R52 port work recorded and it is worth naming: conditionally compiled code that a default build never assembles reads as covered because it is never counted. The answer here is not a merged gcov report — the first two arms are assembly on a target — it is that the union of the two board configurations assembles both arms of all three, and each arm is executed: the T32 trampoline on every silicon run, the A32 entry on every model run, and both console backends by whichever board is running.

A fourth axis existed and was removed for exactly this reason: the test-only fault-continue mode was once an #ifdef in the hypervisor, which meant the shipping path and the test path were different code. It is now a value the image supplies at build time and there is no conditional in the trap vector at all — every build takes the same path and the callee decides. See docs/decisions.md D26.


Reproducing it

./test/host/run.sh coverage

builds instrumented, runs the suite, writes an HTML and XML report, and then enforces the floor as a separate gcovr invocation so that the number being enforced is visible next to the failure.

It deletes the counters first, and that is not tidiness. gcov accumulates: a .gcda left from an earlier run is added to by the next one, never replaced, so a second run over the same build tree reports the union of both and coverage can only ever appear to go up. Measured, not feared: disabling one rule's tests and re-running still reported 100%, and the floor passed. CI never sees it because every run starts from a fresh checkout — which is exactly why it survives, since the machine that gets the wrong answer is the contributor's, checking whether their own change dropped coverage.