Skip to content

xhci: arms for the verdict function that says it was written to be testable - #2305

Merged
bong-water-water-bong merged 1 commit into
mainfrom
lane/xhci-verdict-arms
Nov 12, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
lane/xhci-verdict-arms

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

tools/xhci_boot_capture.sh's verdict function documents its own testability and had no test:

# A function, not an inline block, so the three classifications can be driven on a fixture without a device (the
# same reason every other rule in this tree is header-only/function-shaped: the decision is what can be wrong).

It was written to be testable and then was not tested — the four verdicts only entered the record through a manual drive during engine#823's review, which is exactly the evidence a merge cannot keep. This is that drive, as arms.

The five

control=auto + suspended  -> rc 1, "the keep-awake rule did NOT hold it"    ← the fault #823 named
control=0    + suspended  -> rc 0, "held out of D0 deliberately"
active                    -> rc 0, and NOTHING printed
resuming                  -> rc 2, "a transition that did not settle"
current                   -> rc 3, "is not one of …"                        ← an unnameable state is its own verdict

The silent arm is deliberate. An awake controller prints nothing and returns 0 — the line exists to explain a state, and "healthy" is not one of the things it explains. A verdict file that says nothing about a fault is the failure this whole capture exists to avoid, so the silence is asserted rather than assumed.

Two things the arms caught in themselves

  • grep -qF "" always matches — the first version of the empty-line arm passed for the wrong reason. The harness now handles the empty needle explicitly.
  • The active arm was wrong before the function was. I expected a line; the function is silent by design. Corrected the arm, not the code — the usual direction for a fixture, and worth saying because the reflex is to "fix" the script.

Verified, both directions

unchanged function                    5 arms pass; ctest `xhci_verdict_line` Passed
the auto/0 distinction broken           rc 1:
  if [ "$_ctl" = auto ] -> if false     FAIL control=auto + suspended is the fault (engine#823): want rc 1, got 0

The capture script runs its own capture at the bottom, so the arms extract the function (sed -n '/^xhci_verdict_line()/,/^}/p') rather than sourcing the script: they assert the decision, which is what the comment says can be wrong, and never the device.

Registration

One add_test row, no build target — a script row, which ci_targets_check documents as free ("A script test names its interpreter … and bash is not an add_executable, so a self-test runner costs nothing here"). ci_targets_check --check rc 0, tmp_path_check --check rc 0, copyright.py --check rc 0 after --fix (the header must be the full Apache paragraph, not the short SPDX form).

…stable

tools/xhci_boot_capture.sh's verdict function documents its own shape:

  # A function, not an inline block, so the three classifications can be driven on a fixture without a device (the
  # same reason every other rule in this tree is header-only/function-shaped: the decision is what can be wrong).

It was written to be testable and then was not tested -- the four verdicts only entered the record through a manual
drive during engine#823's review, which is exactly the evidence a merge cannot keep. This is that drive as arms.

  control=auto + suspended  -> rc 1, "the keep-awake rule did NOT hold it"   <- the fault #823 named
  control=0    + suspended  -> rc 0, "held out of D0 deliberately"
  active                    -> rc 0, and NOTHING printed
  resuming                  -> rc 2, "a transition that did not settle"
  current                   -> rc 3, "is not one of ..."                     <- an unnameable state is its own verdict

The empty-line arm is deliberate: an awake controller is silent, and a verdict file that says nothing about a fault is
the failure the capture exists to avoid, so the silence is asserted rather than assumed. (It also needed its own branch
in the harness -- `grep -qF ""` always matches, which is how the first version of this arm passed for the wrong reason.)

Verified, both directions:

  unchanged function      -> 5 arms pass, ctest `xhci_verdict_line` Passed
  the auto/0 distinction broken (`if [ "$_ctl" = auto ]` -> `if false`)
                          -> rc 1: "FAIL control=auto + suspended is the fault (engine#823): want rc 1, got 0"

The capture script runs its own capture at the bottom, so the arms extract the function rather than sourcing the script:
they assert the decision, which is what the comment says can be wrong, and never the device.
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 370698a — and the finding is the sharpest kind of self-indictment a comment can produce.

"It was written to be testable and then was not tested"

# A function, not an inline block, so the three classifications can be driven on a fixture without a device
# (the same reason every other rule in this tree is header-only/function-shaped: the decision is what can be wrong).

That comment is mine — I reviewed #2232 and praised exactly this property: "It is a function so it can be tested without a device." The property was true, the opportunity was real, and no test was written, so the four verdicts (settled / transition / held / UNKNOWN) entered the record only through "a manual drive during en[gineering]" — which is the same as saying they were never checked after that afternoon.

This is the #2238 class in its purest form: not a self-test that no runner ran, but a function that exists for no other purpose than being tested, with no test. The held/UNKNOWN distinction that is the whole point of that tool — "a state this tool cannot name is a state it cannot vouch for" — was the part most in need of arms, because its correct behaviour is subtle (refusing to count a state as settled), and subtle behaviour is what a manual drive cannot verify twice.

What the arms have to cover, given #823's question

The device question is "which of four ports has the device, on a cold boot with Secure Boot on", and the tool's answer is a per-controller verdict. So the cases that matter are:

  1. resuming/suspending → the boot cannot answer (not "no device"), so the arm must assert the cannot-answer outcome, not merely that the string appears;
  2. suspended + control=auto → the keep-awake rule did not hold the controller — the arm that distinguishes the fix not applying from a held controller;
  3. suspended + control=on → held deliberately, not a fault, and (my ask on xhci_boot_capture: say which of engine#823 two faults the boot shows, not just the states #2232) not a candidate port either;
  4. UNKNOWN → counted apart, and the count must still total the controller count.

Arm 4 is the one I would insist on: a fixture with an unnameable state must produce UNKNOWN and leave the buckets summing to N, because the failure mode this tool exists to prevent — an unclassified controller vanishing from the tally — is invisible unless the sum is asserted.

Two asks

  1. assert the totals, not only the labels — every arm should end with "buckets sum to the controller count", since that is the property the UNKNOWN bucket protects;
  2. carry the fixture in the audit's own shape (four controllers, one in transition) — the same fixture shape xhci_boot_capture: say which of engine#823 two faults the boot shows, not just the states #2232 used, so the test and the tool agree about what a boot looks like, and a future change to the ledger's shape breaks one place rather than two.

And the row's status is unchanged

#823 still needs the owner's four physical plugs — no test can substitute for a cold boot with one device per port under Secure Boot. But the arms make the interpretation trustworthy before that window, which is what this PR is for: so that when the owner does plug, a wrong verdict is a bug and not an ambiguity.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Arms verified by running them — 5 arms pass — and one way the extraction could stop being safe.

ok   control=auto + suspended is the fault (engine#823)
ok   control=0 + suspended is deliberate, not a fault
ok   an awake controller is silent (rc 0, no line)
ok   a transition is neither awake nor asleep
ok   an unnameable state is its own verdict, not a pass
xhci_verdict_line: 5 arms pass

Driving the decision instead of the device is the right call, and the arm set is the right set: the deliberate-suspension case (control=0) is the false positive that would otherwise make the check useless, the transition case is the state a naive power_state != "D0" test gets wrong, and "an unnameable state is its own verdict, not a pass" is the refusal-not-a-guess rule. The header's own admission — a function written to be testable and then not tested, entering the record only through a manual drive — is exactly the kind of honesty that makes a test worth more than its line count.

One hardening, because the extraction relies on formatting. The function is pulled out with:

sed -n '/^xhci_verdict_line()/,/^}/p' "$src" > "$fn"
...
. "$fn"

Two things this depends on that nothing states or checks:

  1. The closing brace must be a lone } at column 0. It is today, so the current behaviour is correct — but if the function is ever re-indented (or a nested block's } lands in column 0), the range runs to end of file, and sed then copies the rest of xhci_boot_capture.sh — including the capture that runs at the bottom — into $fn. [ -s "$fn" ] passes, sourcing $fn executes the capture on the host, and the test becomes a device probe with a passing-looking name.
  2. There is no check that what was extracted is a function, only that it is non-empty.

Two lines close both, without changing the extraction:

[ "$(tail -1 "$fn")" = "}" ] || { echo "FAIL the extracted text does not end at the function's closing brace"; exit 1; }
grep -q "^xhci_verdict_line()" "$fn" || { echo "FAIL the extraction is not the verdict function"; exit 1; }

or extract by brace depth instead of by a regex range. Worth doing precisely because the failure mode would look like a working test right up until it touched a machine.

@bong-water-water-bong
bong-water-water-bong merged commit d7ef101 into main Nov 12, 2026
9 of 10 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the lane/xhci-verdict-arms branch November 12, 2026 21:02
@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

823 - Partially compliant

Compliant requirements:

  • The PR addresses the testability of the xhci verdict function which was part of the analysis for the USB port issue
  • The PR adds tests to verify the behavior of the xhci verdict function under different conditions

Non-compliant requirements:

  • The PR does not directly address the root cause of the USB port functionality issue
  • The PR does not provide a fix or mitigation strategy for the USB port issue

Requires further human verification:

  • The actual USB port functionality issue on the Bosgame BeyondMax board needs to be verified with the changes
  • The impact on 1bit OS users needs to be confirmed
⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Test Coverage for Verdict Function

The test script ensures that the xhci_verdict_line function in tools/xhci_boot_capture.sh behaves correctly under various conditions. It verifies that the function returns the expected exit codes and output strings for different combinations of controller states, which is important for correctly diagnosing USB controller issues.

arm() {  # <bdf> <control> <power_state> <runtime_status> <want-rc> <want-text> <what>
    local out rc
    out=$(bash -c ". '$fn'; xhci_verdict_line '$1' '$2' '$3' '$4'" 2>&1); rc=$?
    if [ "$rc" != "$5" ]; then
        echo "FAIL $7: want rc $5, got $rc"
        fails=$((fails + 1))
    elif [ -z "$6" ]; then
        if [ -n "$out" ]; then
            echo "FAIL $7: want no line, got: $(printf '%s' "$out" | head -1)"
            fails=$((fails + 1))
        else
            echo "ok   $7"
        fi
    elif ! printf '%s' "$out" | grep -qF -- "$6"; then
        echo "FAIL $7: the line does not say '$6' (got: $(printf '%s' "$out" | head -1))"
        fails=$((fails + 1))
    else
        echo "ok   $7"
    fi
}

# The fault my #823 review named: suspended under a control of `auto` means the keep-awake rule did not hold the
# controller, and that is the one verdict a boot must not report as healthy.
arm 0000:0a:00.0 auto D3hot suspended 1 "the keep-awake rule did NOT hold it" \
    "control=auto + suspended is the fault (engine#823)"
arm 0000:0a:00.0 0    D3hot suspended 0 "held out of D0 deliberately" \
    "control=0 + suspended is deliberate, not a fault"
# An awake controller prints NOTHING and returns 0: the line exists to explain a state, and "healthy" is not one of
# the things it explains. Asserting the silence matters as much as the text of the other four -- a verdict file that
# says nothing about a fault is the failure this whole capture exists to avoid.
arm 0000:0a:00.0 0    D0    active    0 "" \
    "an awake controller is silent (rc 0, no line)"
arm 0000:0a:00.0 0    D0    resuming  2 "a transition that did not settle" \
    "a transition is neither awake nor asleep"
arm 0000:0a:00.0 0    D0    current   3 "is not one of" \
    "an unnameable state is its own verdict, not a pass"

if [ "$fails" -ne 0 ]; then
    echo "xhci_verdict_line: $fails arm(s) failed"
    exit 1
fi
echo "xhci_verdict_line: 5 arms pass"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant