Skip to content

test(sim): commit the differential harness behind the DMA write-address fix (Refs #2348) - #2379

Merged
gHashTag merged 1 commit into
masterfrom
sim/2348-dma-write-address-harness
Aug 22, 2026
Merged

test(sim): commit the differential harness behind the DMA write-address fix (Refs #2348)#2379
gHashTag merged 1 commit into
masterfrom
sim/2348-dma-write-address-harness

Conversation

@gHashTag

Copy link
Copy Markdown
Owner

What

Commits the differential testbench behind PR #2345 as sim/tb_bitnet_dma_write_address.v.

#2348 records that four merged RTL fixes cite differential simulation as their primary
evidence and none commits the instrument. This lands one of the four. The claim was
never in doubt — an independent audit reproduced all four — the defect is that the
evidence was unreproducible from the repository.

Refs #2348
Refs #2003

Path

sim/, not the bootstrap/tests/rtl/ the issue suggested. sim/tb_bitnet_request_overflow.v
(#2351) is the one harness already committed, and one convention is worth more than a
better-named second one. Same directory, same shape, same README structure.

The published number reproduces

PR #2345's body claims: "OLD writes beats to addr 1..4, mem[0] never written; NEW
writes 0..3"
. First run, before any change to the harness:

== CASE 1: 4-beat read transfer (length=32 bytes) ==
    old: writes=4 addr_range=1..4 done=1
    old wrote: 1(=0),2(=1),3(=2),4(=3)
    new: writes=4 addr_range=0..3 done=1
    new wrote: 0(=0),1(=1),2(=2),3(=3)

(=N) is the payload: beat N lands at address N+1 in the old rendering and at address N
in the new one.

Reconstruction needed no build

bootstrap/src/bitnet_dma.rs has no use statements and no crate:: references, so the
emitter compiles standalone under rustc at any revision. No cargo build, no target
directory. The pre-fix and post-fix renderings come from PR #2345's own base
(cb1f0d4eb980) and head (4db5729b1817); diffing them shows exactly that PR's four
hunks and nothing else. Commands are in sim/README.md.

The harness also passes against current master, whose later waves renamed beat_index to
word_index — every check is made at the module ports.

Three bars

TRUE — the harness compiles under iverilog -g2005 and the numbers above are the
measured output.

ALIVE — it runs, and its anti-vacuity anchors fire. During development the control
case caught itself proving nothing:

  FAIL harness: control produced no AXI write beats -- the write path was never entered and the control proves nothing

BITING — demonstrated with two planted mutants on the post-fix rendering, one per
guard.

Mutant A, reverting the READ_DATA pairing to local_addr <= local_addr + 12'd1:

  FAIL new lowest address written: got 1, want 0
  FAIL new address 0 written 0 times, want exactly 1
  FAIL new wrote address 4, one past the transfer
RESULT: FAIL (14 errors)

Mutant B, deleting only the IDLE re-arm of the index — case 1 still passes, and case 2
alone catches it:

== CASE 1: 4-beat read transfer (length=32 bytes) ==
== CASE 2: second transfer, no reset between ==
  FAIL new lowest address on second transfer: got 4, want 0
  FAIL new highest address on second transfer: got 7, want 3
RESULT: FAIL (6 errors)

That separation is the point: case 2 is not redundant with case 1, and it is the only
check constraining the IDLE hunk.

This is reporting, not a gate

Stated plainly because the distinction matters. vvp exits 0 whether the run prints
RESULT: PASS or RESULT: FAIL — measured both ways, matching tb_bitnet_request_overflow.v.
And nothing executes it: cargo test -p t27c is invoked by no workflow (removed by #2292)
and fpga-build.yml never calls vvp (#2241). Neither is changed here#2348 is
explicitly about the artefact existing in the tree, and wiring sim/ into CI overlaps
#2241/#2275.

The harness found three defects in itself first

All three are commented at the point of fix, because each is a way a testbench can look
right and measure nothing:

  • local_addr is an unsigned 12-bit port; comparing it against an integer seeded to -1
    promotes the expression to unsigned, so the high-water mark never advanced and every
    range read 1..-1.
  • A [255:0] label argument silently truncated a 38-character assertion name from the
    left (FAIL ghest address on second transfer).
  • The start pulse raced the DUT's own sampling at the same timestep, so the transfer
    never began and three cases reported zeros.

The published numbers reproduced before any of these were corrected, so none of them
was fitted to an expected answer.

Scope

One of four. #2337 (#1977), #2340 (#1985) and #2344 (#2006) still have no committed
harness; #2348 stays open.

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-22 01:02:05 UTC

Summary

Status Count
Total Open PRs 4
PRs with Failing Checks 2
PRs with All Checks Green 2
READY 1
FAILING 2
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=65f033d04125 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@gHashTag
gHashTag enabled auto-merge (squash) August 22, 2026 01:02
…ss fix

PR #2345 cited a differential simulation as its primary evidence and did not
commit the testbench, so the claim was true but unreproducible from the tree.

sim/tb_bitnet_dma_write_address.v elaborates the pre-fix and post-fix renderings
of dma_controller in one simulation and drives them from identical stimulus,
following the sim/tb_bitnet_request_overflow.v precedent from #2351. The
published numbers reproduce: old writes 1..4 and never writes address 0, new
writes 0..3.

Reporting only -- vvp exits 0 on PASS and on FAIL, and no workflow runs it.
Wiring sim/ into CI is out of scope here and unchanged.

Refs #2348
Refs #2003
@gHashTag
gHashTag force-pushed the sim/2348-dma-write-address-harness branch from 1e40ac1 to b93990a Compare August 22, 2026 01:09
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-22 01:09:50 UTC

Summary

Status Count
Total Open PRs 3
PRs with Failing Checks 1
PRs with All Checks Green 2
READY 1
FAILING 1
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=65f033d04125 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@gHashTag
gHashTag merged commit ebdf9f6 into master Aug 22, 2026
18 of 20 checks passed
gHashTag pushed a commit that referenced this pull request Aug 22, 2026
…ecipe (Refs #2348)

Adds a README section per harness in the style #2379 established, plus the
shared emit recipe: every BitNet emitter compiles standalone under `rustc`
with a four-line driver, no cargo and no target directory, because the only
`use` in any of them is `use super::*` inside `#[cfg(test)]`.

Records the base/head shas each harness was rendered from, and the three
instrument faults found while reproducing the published claims -- a
posedge-sampled observer trailing the design by a cycle, an observer racing
the stimulus that drove `start_prefetch`, and `first_chunk`/`last_chunk`
having no reset in either rendering. Each was a fault in the instrument; every
published number reproduced once the instrument was corrected.

Refs #1977, #1985, #2006
gHashTag added a commit that referenced this pull request Aug 22, 2026
) (#2381)

* test(sim): differential harness for the zero-count layer_sequencer hang (Refs #1977)

`layer_sequencer` never left RUN when asked for zero work: both terminators
are `index == count-1` against an unsigned port, and the bare literal 1 widens
each subtraction to 32 bits, so a zero count borrows to 32'hFFFFFFFF while the
index zero-extends.

Elaborates the PR #2337 pre-fix and post-fix renderings in one simulation.
Reproduces the published numbers: 200,000 cycles with no `done` and
`neuron_id` reaching exactly 50,000, plus six non-zero controls that are
identical on every output every cycle.

Reporting, not a gate: `vvp` exits 0 on FAIL as well as PASS.

Refs #2348

* test(sim): differential harness for the stale prefetch_done level (Refs #1985)

`weight_prefetch_ctrl` documented `prefetch_done` as a one-cycle pulse but
cleared it only inside the start guard, leaving it asserted for the whole idle
gap. A requester sampling it in the cycle it raises `start_prefetch` reads the
previous transaction's completion.

Elaborates the PR #2340 pre-fix and post-fix renderings in one simulation.
Reproduces the published numbers: t2 sampled_done OLD=1 / NEW=0, done_rises
2/2, we_count 4/4. A second case varies the idle gap to separate level from
pulse: old's high-time grows with the gap, new's does not.

Reporting, not a gate: `vvp` exits 0 on FAIL as well as PASS.

Refs #2348

* test(sim): three-way harness for the latent local_we default (Refs #2006)

#2006 defaults `local_we` low ahead of the case. The pre-fix and post-fix
renderings are observationally IDENTICAL: every reachable path already drove
the strobe, and the states that never mention it are never entered with it
high, because READ_DATA's only exit is DONE_ST.

A harness that passes by finding no difference proves nothing, so this one is
three-way: A = pre-#2006 (PR #2344 base), B = #2006 (PR #2344 head), C =
#2006 + #2003 (PR #2345 head). B and C are consecutive revisions -- #2344's
head rendering is byte-identical to #2345's base -- so one comparator sees all
three. A vs B must be identical; B vs C must differ. Same comparator, same
stimulus, same run, same 293-bit vector of every output port. Putting B in the
C slot makes the run fail rather than certify its null result.

Measured: A vs B 0 mismatching cycles, B vs C 266, over 373 cycles and seven
phases. Deleting READ_DATA's `end else local_we <= 1'b0;` from both renderings
makes A emit 22 local writes against B's 18 -- latent today, a real backstop
the moment an arm stops clearing the strobe.

Reporting, not a gate: `vvp` exits 0 on FAIL as well as PASS.

Refs #2348

* docs(sim): document the three new harnesses and the standalone emit recipe (Refs #2348)

Adds a README section per harness in the style #2379 established, plus the
shared emit recipe: every BitNet emitter compiles standalone under `rustc`
with a four-line driver, no cargo and no target directory, because the only
`use` in any of them is `use super::*` inside `#[cfg(test)]`.

Records the base/head shas each harness was rendered from, and the three
instrument faults found while reproducing the published claims -- a
posedge-sampled observer trailing the design by a cycle, an observer racing
the stimulus that drove `start_prefetch`, and `first_chunk`/`last_chunk`
having no reset in either rendering. Each was a fault in the instrument; every
published number reproduced once the instrument was corrected.

Refs #1977, #1985, #2006

---------

Co-authored-by: Claude <claude@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant