Repository navigation
Merge main into #1981: the five python rows main added move into the family file (Refs #1979) - #2126
Closed
bong-water-water-bong wants to merge 100 commits into
Closed
bong-water-water-bong wants to merge 100 commits into
bong-water-water-bong wants to merge 100 commits into
Conversation
…mit ours (Refs #1945) (#1964) A mirror commit on no AMD branch was always reported as a commit of ours to move to 1bit/hrx-vulkan-patched. On 2026-11-10 it was AMD's own #155: AMD had re-committed its series on a newer base and dropped the old line. The script now checks AMD's pin for a commit with the same author, author date and subject, and names the two cases apart. Test with local repos. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
… clang build has its HRX0 again (fixes #1926, #1942) (#1960) Four fork commits past 2d62cc7af: - 7fb31383b (#282) the fused Q decode kernel compiles at every head size, not only a power of two (#1926) - c47eae4a7 (#279) the host term is read before it is classified, so a clang build registers HRX0 (#1942) - 69ee6f50e (#281) IQ3_XXS routed experts take the workgroup-table decode by default (#572) - a5dc597b8 (#278, carries #277) a pool refusal at compute time is ALLOC_FAILED, with device-free tally arms (#1618, #1823) Measured at a5dc597b8 on gfx1151 (amdclang build, gpu-hold): --list-devices prints HRX0; test-hrx-attention-q-decode-head-size 15 cases, 0 failures; Sarashina2.2-3B bf16 tg128 26.49 / 25.92 tok/s, stories15M Q8_0 tg128 317.77 / 338.54 tok/s. registry/architectures.json: the recorded source pin only. docs/hrx.md: the #1926 fix. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…er (engine#1781 gate 2) (#1958) The NPU draft loop accepts a draft when it equals the base model's argmax. That is exact for greedy decode and wrong for a sampler: it always emits the argmax, 0.55 away from p on p=[.45,.35,.20], q=[.10,.80,.10]. #1781's contract (decided on the issue) is that the output follows the target's distribution, with greedy identity as the temperature-0 case. npu/mtp/spec_accept.h is that rule, pure and header-only: accept a drafted token x ~ q with probability min(1, p(x)/q(x)); at the first rejection emit a token from normalise(max(0, p - q)) and stop the chain. The caller supplies the uniforms, so a run is reproducible from its seed. tests/spec_accept_test.cpp (ctest spec_accept, 21 arms, fixed seed): the temperature-0 identity; a deliberately wrong drafter whose output is within 0.0016 of p over 200k draws at acceptance 0.5497 (sum(min(p,q)) = 0.55); the greedy rule failing the same arm at 0.55; good, identical, disjoint, one-hot and 257-token drafters; the zero-mass and out-of-range edges; the chain's stop-at-first-rejection and two-draws-per-position properties. An always-accept mutant fails 7 arms. Not wired into a serve: that is the rest of #1781. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…package is in forms we already read (engine#1780) (#1957) A DFlash draft package (FastFlowLM 1.0.12's two-package tags) is config.json (model_type dflash) plus one Q4NX container. The thread on #1780 stopped at two byte-identical readings of the projections (q4_1 4096x2560 against a half-float 1280x2560) and at a claim that the BF16 labels are really F16. Both are settled from the bytes: - the I8 [tiles, 5120] projections are the ordinary q4_1 tile grid: all 35 have bytes == (N/32)(K/256) x 5120 for the config's N x K, 10,240,000 groups have a finite positive scale, and minimum/scale is -7.59 .. -7.33 per tensor (a main-model container our forward serves reads -7.41 the same way). The half-float reading has 25.7 % of values above 100 in magnitude. So head_dim is 128, as the config says. - the BF16 labels are bf16: the 'tight as half-float' pattern is also present in a main model that runs correctly with the bf16 reading, because half-float folds bf16's exponent range into one octave. tools/dflash_draft_info.py checks a package against its own config (tile-grid sizes by role, tile-header sanity, the container's scalars, fc and norm sizes, no unexplained tensor) and fails closed; 19 of 19 on the Bolt draft in 3.4 s. --selftest builds a synthetic package with eight negative controls (ctest tool_selftest_dflash_draft_info). Stdlib only. docs/amd-drafters.md records the answers and what is actually missing: the draft's forward, not a format. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…) (#1948) A pack that declares /lm_head/MatMul_Q4 at block_size 32 (the Ryzen AI 1.9 Qwen-1.5B packs) keeps the head after the tail envelope in 64-row units of 2,560 + 2*(K/32)*640 bytes, not in the block-128 tile. read_block32_head() reads that layout; import_pack() takes it when the layout checks hold and, on a tied pack, the result correlates above 0.99 with model.embed_tokens. Otherwise a tied pack keeps the synthesis (and the note says why), and an untied one is refused with the reason. Measured on Qwen-2.5_1.5B_Instruct_rai_1.9.0_npu: 2,374 units of 64,000 B, corr 0.996185 against the embedding over all 151,936 rows, row-shifted control 0.0554. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…t stays inside its K*N image (Fixes #1994) (#1999) host_gemm_i8 and the i8 LM head pack UNPADDED images (K*N bytes) with blocked=1 and read them back with blocked_index(), which addresses whole 64x128 tiles. On a geometry that is not whole tiles, both the packer and the MAC run past the vector. Phi-3-mini's last vocabulary block (K 3072, N 7488) overruns by 196,096 B; a body width of N 8512 overruns by 130,560 B. Both sites now pick blocked = q4nx_layout::blocked_fits(K, N), record it in HostI8Tensor::blocked, and read through the new q4nx_layout::image_index(). Row-major holds the same codes and scales, so the int32 sums are unchanged. Every whole-tile geometry (the measured 11008 body and 2688 head block) keeps the blocked fast path. tests/npu_blocked_layout_test.cpp: test_unpadded_image packs into K*N plus 256 KiB of guard bytes at 3072x7488, 2048x8512 and 100x128 (row-major) and 4096x2688 (blocked). It checks that the guard is intact, every code reads back inside K*N, and forced blocked really leaves the buffer. Mutation: forcing blocked=true fails 3 checks on each unaligned shape. docs/npu.md describes the rule. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…/k gains, naming both witnesses (engine#1985) (#1995) A regression, not a new bug. `Qwen3-1.7B` imported with **0 gaps** on yesterday's main and refuses with **280** on today's, on the same complete pack: yesterday attention_payload {o_proj_k: 2048, qk_norm: true, tiles_measured: 64} today attention_payload {o_proj_k: 0, qk_norm: false, tiles_measured: 0} first refusal: model.layers.0.self_attn.q_proj.weight: attention payload holds 64 tiles; q/k/v/o at 64 ... rows each need 96 `qk_norm: true -> false` is the tell: the pack has the two per-head q/k sections between q/k/v and o_proj, so its block is a 4-header one -- and it is now read as 2-header, which takes the 2-header arithmetic and refuses every attention projection. The cause is #1838's early return: `if (!g.hdr.empty()) return g;` trusts the epsilon-anchor vote's list, which for this pack recorded ONE in-block header. That guard was right for a vote that found the right list and wrong for a vote that found only one on a block that has three -- the #1831 defect coming back through the vote instead of through the residue step. Neither witness wins, so the reader refuses and names both: the epsilon-anchor vote recorded ONE in-block header, and this pack declares use_qk_norm, whose per-head q/k gains live in the two sections a 2-header block does not have (engine#1985). The two witnesses disagree and neither is being preferred `header_witness_refusal(vote_hdrs, declared_qk)` is the whole decision in one testable place, and the three cases it must NOT fire on are as important as the one it must: a three-header vote (a correct 4-header pack), a one-header vote on a pack that declares nothing (a genuine 2-header family), and **no vote at all** (0 -- that is the residue step's case, not a disagreement, and refusing there would take `Llama-3.2-1B` out). Arms: 8, host-only. Proved they can fail -- making the predicate always return "" right-fails 5 arms. Suite 191/191. (The first run after the rebase showed 3 "Not Run" -- `serve_degenerate`, `q4nx_cache`, `npu_answer_parser` -- targets an incremental build had not built; building them gives 100%. That is the fourth time tonight a suite result was about the build rather than the code, and the reason "1 failed" must always be read as "which target, and was it built".) This does not *fix* Qwen3-1.7B: it stops the silent misclassification and says both witnesses. The 4-header read itself is #1934/#1969's measurement (what the 640 B `qk_rms_packed` gap actually holds), and the Qwen3-4B/8B rows should be re-run against this head rather than trusted from before the window. Refs #1985, #1831, #1838, #1889. Co-authored-by: t <t@e.com>
…p forward cannot use it (refs #1740, #1868) (#1991) engine#1868 refuses --npu-opt when the build links no private route. With an add-on linked there IS one, so the option was accepted -- and then bypassed by the CONTAINER TYPE: a model-generic Q4NX container is routed to the per-op forward before the private route is consulted, so the option was ignored again, one level in. Found by running W6: 'serve -m <container> --npu-opt dx=<elf dir>' asked for the fused route, got per-op, and produced *identical* numbers to the per-op baseline (12,010 vs 12,169 ms for 24 tokens, same reply) -- which reads as 'no gain' rather than 'wrong route'. The engine's own log line says which forward it chose, but nothing said the option had been dropped. Now the per-op branch refuses by name when the caller asked for a private-route option: --npu-opt dx was given, but <container> is a model-generic container, which serves through the per-op forward: the option has no effect here (it applies to a private route's own model directory). Drop it, or serve the model directory that route expects. Verified both directions on the real container with an add-on-enabled build: with the option -> exit 2 and that message; without it -> the per-op forward still starts and prints 'serving it through the per-op forward' as before. Co-authored-by: pi-inferzo <pi-inferzo@1bit.gg>
… in all-NaN logits (fixes #1539) (#1987) Carries fork #284 and #285 (the routed-expert and dense matmul kernels keep their activations inside the half-precision range) and #283. Adds the docs/hrx.md section for the fix; the first part of #1962. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…'s registration is a new file (Refs #1979) (#1983) * build: every cmake/tests-*.cmake is included by a glob, so a new test's registration is a new file (Refs #1979) A test-adding PR appends a row to CMakeLists.txt, and that file changed 37 times in the 24 h before #1979 was filed, so the row is the first part of the PR to stop applying. The family-file include from #1696 becomes a glob over cmake/tests-*.cmake at the end of the file, after every target and after find_package(Python3). A registration added as a new file there touches no shared line. No existing row moves: ctest --show-only=json-v1 gives the same 186 tests, commands and properties before and after. tools/ci/ci_targets_check.py's glob reader wanted <dir>/*.cmake and matched nothing for a prefix glob; it is widened, with an arm that fails on the old pattern. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * build: the glob skips a tests-*.cmake file CMakeLists.txt already includes by name So a family file moved out with its own include() line (#1981's shape) and the glob can land in either order: without the skip, the second include stops the configure with 'add_test given test NAME ... which already exists'. ci_targets_check reads such a file once. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * build: the glob's skip matches an include() line, not the same text in a comment Review of #1983 at 8dc5a26 (two CHANGES): the skip was a substring search over CMakeLists.txt, so a comment such as "# was include(cmake/tests-gpu-hold.cmake)" made the glob skip the file while nothing included it. The seven gpu_hold_* tests left the configuration (186 -> 179) and ci_targets_check stayed green. The skip is now a line-anchored match, the rule ci_targets_check.py's INCLUDE reads by, with the dots in the name escaped and the CMAKE_SOURCE_DIR and quoted spellings recognised. tests/registration_glob_test.sh cuts the block out of the real CMakeLists.txt and configures it in scratch projects (9 cases). It is registered from cmake/tests-registration-glob.cmake, a file nothing names, so it is also the first registration that arrives through the glob. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…p (Refs #1956) (#1988) The sweep orders entries by the entry directory's mtime, which nothing moved after the entry was written, so the order was by creation: a container served every day went before yesterday's one-off. A hit now touches its entry (repacked GGUF, lane layout, device-form conversion). ONEBIT_Q4NX_CACHE_MAX_GIB was multiplied by 2^30 unchecked: 2^34+1 came out as a 1 GiB bound, and a value that is not a number turned the sweep off. The bound is now clamped, and a non-number gets the default. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…norm (refs #1503, follows #1822) (#1977) #1822's instrument, re-applied on current main with its open review answered: the variable is read once per process, the mean/absmax arithmetic and the line's text are tested, and the line is printed on the host arms and the device arm of both npu_gemm() and npu_gemm_rows(), so a host run and a device run can be compared line for line. Non-finite values are counted instead of vanishing from absmax. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
… genre, and match both layer-name spellings (engine#1786, item F1) (#1974) * tools/flm: report a FastFlowLM NPU2 model source's container without a device (engine#1786, item F1) F1 of the FLM 1.0.12 parity work is 'every FastFlowLM/*-NPU2 repo imports and serves on --device npu'. The first row showed the packs are a genre our importer does not know: model.q4nx is a safetensors container (8-byte LE header length + JSON of name -> {dtype, shape, data_offsets}), the tensor NAMES are the model's own HF names (no graph, no model.onnx), and the pack holds only BF16 (norms/embedding) and I8 (matrices, packed -- down_proj is [640,1280] for a 640-wide FFN). This tool reads the header only: no weights, no device, no runtime. It exists so each of the 59 rows in the board's FLM NPU2 section can be characterised in one command and the importer's work list is a fact rather than a guess. Measured on FastFlowLM/Gemma3-270M-NPU2 (0.54 GB): header 26496 B, 237 tensors, payload 0.503 GB, dtypes {BF16: 110, I8: 127}, 18 layers, metadata keys [] (empty -- this pack has no __metadata__ block). * npu-pack: read a FastFlowLM NPU2 model source's container (engine#1786, item F1) The first row of the F1 sweep showed why the importer refuses every NPU2 pack: its genre is an ONNX graph plus external data, while an NPU2 model source is a safetensors container named model.q4nx (8-byte LE header length, then JSON of name -> {dtype, shape, data_offsets}). Measured on FastFlowLM/Gemma3-270M-NPU2: header 26496 B, 237 tensors, payload 0.503 GB, dtypes {BF16: 110, I8: 127}, 18 layers, no __metadata__. This is the reader for that container, header-only -- no weights, no device -- exported so the half that can be wrong silently is testable without a pack. Each failure names what was wrong, because the caller's next move depends on which: a truncated header must not read as "a pack with no tensors". Arms in the existing registered test (no new target): the real two-tensor shape (BF16 vector + I8 matrix at twice its logical width), __metadata__ skipped rather than counted, a truncated header named, and an implausible header length refused by the number so a mis-typed length cannot make the caller allocate it. Verified by -fsyntax-only of pack_import.cpp and the test against a configured tree's json dep; NOT built or run: disk is at 146 GB, below this repo's 150 GB floor and the 175 GB build threshold. The syntax check found one real defect already (my ContainerTensor collided with the existing ONNX-role type of that name; renamed Q4nxTensor). Next: wire it into plan()/info so an NPU2 pack reports its inventory instead of refusing for model.onnx, then import -> --device npu -> known-answer prompt -> tok/s under npu-hold. * npu-pack: recognise and report a FastFlowLM NPU2 model source instead of refusing at the first byte (engine#1786, item F1) Reading the container header (previous commit) is only half of it: plan() then went on to open a graph the pack does not have, so `npu-pack info` answered a 0.5 GB model source with "cannot open <pack>/model.onnx" -- a missing file on a pack that has no graph by design. plan() now detects the genre first (model.q4nx present, no resolvable graph file), reads the container, and records the census on the Plan: tensor count, payload bytes, a dtype map, and a note saying what the genre still needs. manifest() reports it under "container", so info prints it and a test can pin it. Measured on FastFlowLM/Gemma3-270M-NPU2: 237 tensors, 0.503 GB payload, {BF16: 110, I8: 127}. The note is deliberately explicit that the pack is RECOGNISED AND REPORTED, NOT IMPORTABLE: reading a container as a weights source is the container-genre plan this item still owes, and a plan that quietly listed 237 tensors as importable would be worse than the refusal it replaces. Syntax-checked against a configured tree's json dep; still not built or run (disk 149 GB, under the 175 GB build threshold). * npu-pack: pin the container genre's report, including that it is not importable (engine#1786, item F1) The plan() detection had no arm, which for this change is the difference between a report and a claim: a plan that listed a container's tensors as importable would be worse than the refusal it replaces, because it would assert a compute mapping this importer does not have. The arm builds a pack directory that IS a container and has NO graph, then checks plan() reports the tensor count, the payload the offsets imply, both dtypes, and a note containing "not importable" -- and that manifest() carries it under "container", which is what npu-pack info prints. Syntax-checked against a configured tree's json dep; still not built or run (disk 148 GB, under the 175 GB build threshold). * npu-pack: fix two bugs in my own container arms now that they build and run (engine#1786) Built npu_pack_import_test for the first time and ran it: two of my four arms failed, and both were the arm's fault, not the reader's. Worth recording precisely because the reader was right either way: - the payload arm expected 819200 but the reader returned 820480 -- and the reader was correct: the sum of the data_offsets SPANS is 1280 (first tensor) + 819200 (second). I had summed only the second span, i.e. the arm asserted a number I had derived by hand instead of the number the offsets imply. - the implausible-length arm left its std::ofstream alive while the reader opened the same path, so the reader saw an empty file and reported "shorter than the 8-byte header length" -- the arm was testing its own missing flush. Scoped the writer so it closes first. With those fixed: all npu-pack checks pass (pack leg SKIPPED, no pack directory given), which is the whole suite, so the container reader and the genre report are now verified on this box rather than only syntax-checked. Also: the two failures are a small demonstration of the night's recurring rule -- a check that reports a number you cannot re-derive is a check you have to run to trust. * npu: the per-layer prefix has two spellings, so a failed lookup retries the other (engine#1786, item F1) Three of the fifty-eight FastFlowLM NPU2 model sources spell the per-layer prefix `model.layer.` (singular) where this forward addresses `model.layers.` (measured by header census: engine#1786). Every other tensor in those containers matches by name, so the pack would miss its layer tensors by spelling alone and the failure would look like a missing tensor rather than a naming difference. model_tensor_by_name() now retries the lookup with the other spelling AFTER an exact match has failed, so a container that carries both is unaffected -- the exact match still wins. The rewrite itself is model_tensor_name_alt(), a pure static-inline helper in model.h: it swaps the leading prefix once and leaves everything after it byte-identical (a global replace would corrupt a tensor whose name carries the same text further along, and a near-miss prefix like `model.layersX.` must be left alone). Five arms in npu_forward_loader_test (which compiles model.c and includes the header): both round trips, a model-level tensor with no alternative, the near-miss prefix, and a too-small buffer reporting no alternative rather than truncating. Built and run on this box: all cases ok. The arms earned their keep immediately: the first version skipped the SOURCE prefix using the TARGET's length (12 vs 13 characters), which produced `model.layer.s.0.…`; two of the five arms failed and named it. --------- Co-authored-by: bcloud <bcloud@local>
…ver the current one (Fixes #1907) (#1952) * npu-pack: sweep every fconst scratch directory, and stop the test suite leaking into them (Fixes #1907) Since #1801 the canonical copy is written to ~/.cache, and the sweep ran only in the directory being written, so copies older binaries had left in /tmp (tmpfs, i.e. RAM: 17.0 GiB measured) were never reclaimed. The sweep now covers every candidate directory, creating none. With ONEBIT_FCONST_SCRATCH set, only that directory is swept, so a run with an override never removes another lane's files. The test suite was the source of 179 of the 188 /tmp files: test_fconst_scratch_order() left HOME=/home/x with TMPDIR and the override unset, so later crafted imports fell through to /tmp, and the earlier ones wrote into the real ~/.cache. It now restores the four variables, and main() gives the run its own scratch dir, removed at exit. docs/npu.md: where the copy goes, and what the sweep removes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * npu-pack: never sweep the current directory, and test the sweep through the importer (#1915 review, engine#1907) - sweep_fconst_scratch_dirs skips the current directory (compared on canonical paths) unless it is the directory being written, and sweeps a directory listed twice once - test_sweep_is_wired runs import_pack() with an abandoned foreign copy in the scratch directory and asserts it is gone; with an override set, the cache directory's copy must still be there - boundary arms: a file that is not fconst19 scratch is kept however old; the cwd under three spellings; the cwd when it is the written directory - the 6 h constant says what it protects Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…ixes #1467) (#2011) One fork commit past fa533bd21: 50d28600a (#286), the multi-section rotation in the two generic rotation kernels. With fa533bd21's #283 (already pinned) Qwen3-VL-2B decodes on HRX0 again and its text graph is 2-3 splits instead of 114. Measured at 50d28600a on gfx1151 (amdclang build, gpu-hold): test-backend-ops ROPE 166/166; text identical to the CPU for 12 greedy tokens; llama-bench pp512 6481 / 6532 tok/s, tg64 133.7 / 135.5 tok/s; llama-server, projector on HRX0, first image request 2.99 s (prompt_ms 2782) against the issue's 16.97 s. registry/architectures.json: the recorded source pin only. docs/hrx.md: the #1467 section. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…e, not an arithmetic term) (#2010) * npu-pack: say what a rule row's `prefix` actually is (a twin-diff byte, not an arithmetic term) The table's comments described `prefix` as `64 + 2*hidden + 4,352` (or `64 + 2*hidden` on the families with no QK-norm term). It is not: `tools/fconst19_measure.py:104` walks prefix = 0 while b0[prefix] == a0[prefix]: prefix += 1 on the 1.9 block and its 1.8 twin, so `prefix` is the FIRST BYTE AT WHICH THE TWO DIVERGE, and it is a measurement of that pair and nothing else. The arithmetic agrees on every row measured so far, which is exactly why the mistake is easy: it reads as a rule derived from the geometry when it is a coincidence of the packs that happened to be measured. I reviewed three of those rows myself and "verified" the prefix against that formula, which confirmed the coincidence and not the rule. Correcting the comments is the cheap half of the fix; the expensive half is the row someone derives from it next. A stated consequence, in the header: a 1.9 pack with NO 1.8 twin cannot be given a row at all -- there is nothing to diff it against -- so it stays refused by the shape gate rather than being handed an inferred prefix and reported working. `amd/granite-4.1-8b_rai_1.9.0_npu` is the standing example and must stay refused. Two comments that leaned on the formula are reworded to report the observation as an observation (the 2,560 row's agreement with the 1,504... with the 1.7B twin's landing, and the h3584 pair's note that the prefix is *not* separated by that pair). The four commented rows keep their numbers -- a row's prefix was measured; only the explanation of where it comes from changes. `npu_pack_import_test tests/golden`: all npu-pack checks passed. tools/copyright.py --check rc 0. * fconst19_measure: --check-row, so a table row can be held against the pair it claims to describe (engine#2010) The comment fix says what `prefix` is; this makes it checkable. `--check-row npu/pack_import.cpp` parses `kFconst19Rules`, finds the row for the measured geometry, and FAILS when any field differs from what this pair measures -- so a row derived from `64 + 2*hidden + 4,352`, or copied from a neighbour's shape, is caught by the tool that would have measured it. A geometry with no row reports that too: that is the shape gate's answer for a pack with no 1.8 twin, and it should be a named refusal rather than a row someone inferred. Field order follows the struct (prefix, n, rot, tile), not the order the numbers appear in a comment -- I got that wrong twice while writing this, once in the parser and once at the call site, and the arm caught both because a swapped pair still parses. The arm drives the real tool on the synthetic pair the other three cases already build: a row that matches must pass, and the same row with `prefix + 64` must fail naming both numbers (`8320 ... but this pair measures 8256`). tests/fconst19_measure_test.py: all 4 cases pass. --------- Co-authored-by: bong-water-water-bong <bong@users.noreply.github.com>
…not one ldexp() (Refs #1990) (#2008) 2^e_ref depends only on the column, and it was a libm call per element after readback, where no launch phase column counted it: 137 ms of a 630 ms prefill block on the served 27B pack. It is now computed once per column. Same bits (exact power-of-two multiply; out-of-range exponents keep the per-element form): NPU_DEQ_LDEXP=2 counts differing elements, and npu_pb16i_layout_test holds the two forms equal. The [bl] line names the term as deq. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
… I got wrong four times (refs #1934) (#2003) A safetensors `data_offsets` entry is measured from the **start of the data section**, i.e. `8 + header_len` bytes into the file -- not from byte 8. Fetch `bytes=8+off` and you get a slice of a *different* tensor, one header-length away; it has a plausible size, it is finite, and a correlation against it is a correlation against noise. That mistake cost two false conclusions about a merged PR tonight, both stated publicly before I caught them: "the 640 B gap does not correlate with the source model's q/k gains at any offset" and "the container still holds the pre-#1938 two-copies signature", the second of which I had to retract on #1976 when the correctly-fetched tensor gave `mean 1.3705, max 1.703125, min 0` — matching the container to three decimals. The tell was in my own output: a norm gain is ~1.0 and my "HF" values were +-0.03. So this tool does the arithmetic once and makes a wrong read *visible* rather than plausible: ``` $ tools/hf_tensor.py --repo tencent/Hunyuan-7B-Instruct \ --tensor model.layers.0.self_attn.query_layernorm.weight model.layers.0.self_attn.query_layernorm.weight BF16 [128] 256 B (model-00001-of-00004.safetensors) mean +1.370461 min -0.000000 max +1.703125 nonfinite 0 a norm gain is ~1.0 and a weight matrix is ~0: check these before trusting a correlation ``` * reads only the index, the shard's 8-byte length + header, and the tensor's own bytes (three range requests); * prints mean/min/max/nonfinite, because that is what separates "the tensor I asked for" from "a slice of its neighbour"; * `--out` writes the raw bytes for a comparison script, `--json` for one; * `--self-test` drives the rule on a **synthetic** shard with no network: a first tensor long enough that the second's offset is past it, then the same offset applied from the data section (returns the tensor) and from byte 8 (does not). Registered as `hf_tensor_self` so the rule is pinned in every configure; `ctest -R 'hf_tensor|tests_referenced'` -> 3/3. Refs #1934. This is the first half of the `diff-hf` instrument I asked for on that issue; the second half -- decode a *container* tensor through the importer's own decoder and print the per-row regression ratio -- is the next step. Co-authored-by: t <t@e.com>
…ne#1503, stacked on #1948) (#1955) * npu-pack: read the 1.9 block-32 head from the .fconst tail (Fixes #1740) A pack that declares /lm_head/MatMul_Q4 at block_size 32 (the Ryzen AI 1.9 Qwen-1.5B packs) keeps the head after the tail envelope in 64-row units of 2,560 + 2*(K/32)*640 bytes, not in the block-128 tile. read_block32_head() reads that layout; import_pack() takes it when the layout checks hold and, on a tied pack, the result correlates above 0.99 with model.embed_tokens. Otherwise a tied pack keeps the synthesis (and the note says why), and an untied one is refused with the reason. Measured on Qwen-2.5_1.5B_Instruct_rai_1.9.0_npu: 2,374 units of 64,000 B, corr 0.996185 against the embedding over all 151,936 rows, row-shifted control 0.0554. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * npu-pack: import a 1.9 pack whose every MatMulNBits is block 32 (engine#1503) An all-block-32 pack (SmolLM2-135M-Instruct 1.9) was refused at the block gate. Its raw Token .fconst is now read with its own layout: 64-row units of 2,560 + 2*(K/32)*640 B for q/k/v/o/down and the head, and 64-row units of 4,864 + 2*(K/32)*1,216 B holding gate and up together. Offsets come from the geometry and are checked at every epsilon envelope. read_block32_head becomes a wrapper over read_block32_units; the head logic #1948 added is shared by the mixed pack and this path. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * npu-pack: one arm per declared block size; unmeasured sizes are refused by number (#1503) Review on #1955 asked what a pack declaring a block nobody handles does, and for an arm per declaration rather than one for the class. The all-block-32 fixture is now rebuilt with only cache/ops_meta.json changed: every op at 3 / 64 / 256 refused, 'block size along K is N (11 ops)' layers 32, head 128 refused, not the all-block-32 class layers 32, head 64 refused every op at 128 stays off the all-block-32 path and the gate With the selection made to ignore the declared value, all six arms fail (each writes a container through the all-block-32 path). The gate's refusal now says the all-block-32 layout exists and when it applies. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…s / 36 layers), head read from the decode If branch (#2009) * npu: measure the 1.9 Token fconst rule for hidden 4096 / 32 heads / 36 layers (Qwen3-8B) (refs #1740) Measured with tools/fconst19_measure.py against amd/Qwen3-8B_rai_1.8.0_npu_16K: 26,104 of 3,756,627,264 bytes differ (6.95e-04 %), content rather than layout. The header stride 104,350,656 is uniform over all 36 blocks, checked independently of the tool (36 runs of 32 x 00 00 7f ff, first at 13,796,032). The row is identical to the Mistral h4096 row in every field except the layer count, which is the finding: the bundle shape and rotation are a property of the hidden size, not of the depth. It needs its own entry anyway, because select_rule_in() keys on (hidden, heads, layers). Stated in the comment because it is not all seven fields: --probe-wrong separates rot+1 and tile/2 by 14,700x (10.2 % of the file each), but prefix+64 differs in only 32,960 B -- 1.26x the accepted residual and under the 37,566-byte tolerance -- so this pair does not isolate the prefix. The 12,608 rests on the Mistral row and the Qwen3 arithmetic agreeing, as the 3,584 row's prefix does. * npu-pack: read lm_head from the If branch that is the decode path, so the 4096-hidden untied packs (Qwen3-8B, granite-4.1-8b) get their head from the graph (refs #1740) An OGA 1.9 pack splits prefill from decode with `If (Equal(Shape(input_ids), Constant(value_int 1)))`, and the two branches do NOT carry the same tensors. Measured on amd/Qwen3-8B_rai_1.9.0_npu: then_branch holds exactly ONE MatMulNBits and it is lm_head [151936, 32, 64] at offset 15,269,888 in model.onnx.data (311,164,928 B = V*H/2); else_branch holds the 109 layer projections plus an `lm_head.MatMulNBits.qweight.empty` placeholder with no bytes. The walker merged both branches, so the placeholders it saw for lm_head counted as weightless_matmuls and plan() took the "no readable graph head, read the tail" path. On these packs the tail is the envelope alone -- Qwen3-8B's is 8,256 B = 64 + 2*4096, verified by hex (bf16 1e-6 at kNormOff 64, then the norm weights) -- so `want = env + rows*tile` could never equal `have`, and the import refused with "lm_head.weight: the .fconst tail is 8256 B, not the ... B its envelope and head tiles would take, so the head's offsets are not established". The Token .fconst IS the decode path, so the decode branch's node is the one that describes it: - walk_graph now records each matmul's `If` branch (the attribute's own name); - parse_onnx reads the condition rather than assuming: a Constant value_int of 1 makes then_branch decode, anything else else_branch, and a graph without that shape falls back to then_branch with a note saying the rule was assumed; - import_pack picks the head node from the decode branch, keeping the old first-match behaviour as a fallback when only the other branch declares it; - the #1687 refusal reads weightless_matmuls_decode (outside the decode branch), so a prefill placeholder is no longer mistaken for a genuinely absent weight. The total stays for `info`'s census, and both are reported in graph_weights. Pinned by test_external_packed_genre: a crafted graph whose condition compares the shape with 1 must name then_branch decode, and an inverted one must move it to else_branch. The regression arm for the head pick (decode branch taken, inverted convention falls back, flat graph unaffected) was checked standalone. Not run: the pack itself -- this tree's submodules are not checked out, so libonebit_npu cannot be built here; the changed regions pass -fsyntax-only and the test compiles. * npu-pack: decide the decode If branch by which branch holds the weights, not only by the condition's constant The condition `Equal(Shape(input_ids), Constant(1))` is the pack's statement of which branch is one-token; the arithmetic is what must hold. A pack that inverts the condition without moving its weights answered the wrong branch, and the head vanished. Now the branch that actually declares MatMulNBits nodes wins when only one of them does, and the condition decides only when the graph is ambiguous (weights in both branches). branch_note names which rule was used and, when they disagree, how many matmuls each branch has. Also fixes the crafted test: it wrote model.onnx while this fixture's genai_config.json names eager-mm.onnx, so the arm was measuring the fixture's own graph rather than the crafted one -- and the graph needed its embedding tensor to be plannable at all. * npu: the integer arm's operands come from the engine's packer, not a private re-quantisation The arm was merged in #1809 with a quantisation rule of its own: per 128-K group, int4 codes in [-8, 7]. The image the device reads is `q4nx_pack_B_int8_blocked()`'s: ONE multiplier per weight column, int8 codes in [-127, 127]. Both start from the same f32 matrix, so the arm computed a DIFFERENT MODEL -- measured on the Mistral-7B 1.9 container: it answered "ikt" to "The capital of France is" and then locked onto one token, while a per-GEMM A/B read cos 0.9999 against f32 on the same tensors. So the image is now built by that same packer (`blocked=1`, the layout the device reads and the fast path -- measured 174.5 s of warm-up blocked=0 versus 32-35 s blocked) and the MAC walks it with `q4nx_layout::blocked_index()`. Measured after, on current main: warm-up 35.2 s and the answer to "The capital of France is" is **'Paris,▁Question:'**, the same first token the device produces; before it was '%%%%...' / '▁/******/'. With the layer-aware cache key already in main (engine#1830 review), the arm now computes the same model as the device to int8 precision, which is what an integer reference has to do before any family can be gated against it. `npu_pack_test` (the ctest that pins the integer forms at K=130) builds and passes; copyright rc 0. * npu: keyed dumps settle it -- the device's weight scales are BIT-IDENTICAL to the arm's (384/384) Dumps were paired by index first and that was meaningless: the two sides reach the GEMM a different number of times (1024 arm dumps against 384 device dumps on the same request). Keyed by (layer, N, K, ordinal) they pair exactly. Result, one token, Mistral-7B 1.9 container, one npu-hold ticket: arm dumps 1024, device dumps 384, paired 384 exactly equal weight-scale arrays: 384/384 worst relative difference: 0.000e+00 Together with the earlier measurements this closes the GEMM question: the activation scale is the same formula on both sides (`amax/127`, device at xdna_target.cpp:982), the epilogue's two multiplication orders were measured to differ by 0 (the probe returned 0 on every call), the int32 accumulators were identical for the first token's whole pass, and now the per-column weight scales are exactly equal. So the device's GEMMs are bit-exact against this arm. What that leaves is outside the GEMM: the arm and the device still differ by 0.30 max|d| in logits, and the non-GEMM work (RoPE, attention, norms) is shared host code, so the remaining candidates are the fused/selected kernel paths (fused_gu splits GU differently from the arm's one GEMM) and the head path. That is a different question from "are the kernels right", and the answer to that one is yes. Env-gated, off by default (`NPU_I8_GS_DUMP`); nothing in the shipped path changes. * npu: the integer head, and the gate it unlocks -- device logits BIT-EXACT over four steps The head was the last place the host arm and the device computed the same tensor differently: in host mode `run_lm_head()` is a plain f32 matvec ("No I8Ctx is touched") while the device reads a packed quantised table (`lm_head_packed_`, `run_lm_head_gemv`). That was the whole of the remaining `max|d| = 0.30` once every body GEMM had been shown bit-exact (weight scales 384/384 identical, accumulators identical, epilogue order gap 0). The device packs its head tiles with `lctx.packB_into(...)` -> `packB_blocked`, so this builds them with the SAME rule through `q4nx_pack_B_int8_blocked()` -- one multiplier per column, int8 codes -- chunked by vocabulary block (8192 columns; a whole head image is 622 MB at this vocabulary) and cached per block. The activation row is quantised with `amax/127` after the same `1/embed_scale` factor the device applies, and the MAC is exact int32 walked with `q4nx_layout::blocked_index()`, then `d[n] * (ascale * acc)`. Measured, one npu-hold ticket, device vs this arm, logit dumps compared per step: step 0: argmax 3916 == 3916 cos 1.000000 max|d| 0 step 1: argmax 278 == 278 cos 1.000000 max|d| 0 step 2: argmax 28725 == 28725 cos 1.000000 max|d| 0 step 3: argmax 22478 == 22478 cos 1.000000 max|d| 0 BIT-EXACT over 4 step(s) at max|d| tolerance 0.0 That is TNPU-ALL section 4's bar met literally for this family rather than read down to "at the accumulator": the device's arithmetic and the integer reference agree to the last bit, on every step, for a full request. * npu-pack: read the fused SSMLP/GQO FFN attributes, and the 4096/32/40 shape rule (refs #1740) granite-4.1-8b states its FFN nowhere the reader looked: config.json is literally {}, genai_config.json's decoder block has only hidden_size / heads / layers, and its FFN is not MatMulNBits nodes at all -- it is one `SSMLP` op with gate_K 4096, gate_N 12800, up_K 4096, up_N 12800, down_K 12800, down_N 4096. So the import refused with "the pack states no intermediate_size", which is correct for what it could see and wrong about the pack. Recording those attributes as matmuls lets intermediate_size_from_graph find w by the same witness it already uses for the separate-node spelling (gate and up reading one activation) instead of adding a second rule that could disagree with the first. With the width resolved the pack reaches the shape gate, which had no row for hidden 4,096 / 32 heads / 40 layers. Its 1.9 header is clean -- 40 runs of 32 x `00 00 7f ff`, first at 13,795,392, every gap exactly 107,728,192 -- and its tail is 225,198,144 B, which is exactly 8,256 + 1,568 x 143,616: the h4096 head grid (rows = ceil(100,352 / 64)). So unlike Qwen3-8B the head IS in the tail here, and the bundle/rotation agree with the Qwen3-8B row at the same hidden size and head count. amd publishes no 1.8 twin for this model, so that agreement plus the head-grid arithmetic is the whole evidence for the rotation -- the row says so, and says the prefix inherits the Qwen3 row's own caveat. * docs/npu: say what NPU_INFER_HOST_GEMM=i8 is for, and what it measured The integer arm was only discoverable from the code. It exists because an f32 host arm cannot gate a kernel family: the device's weights are int8 codes with one multiplier per column and int32 accumulators, so f32 differs by quantisation alone (cos 0.9960-0.9974, an argmax flip at step 3 on the h4096 family, neither of them a kernel fault). Documented: what it builds (the engine's own packer and blocked layout, so the reference shares the image rather than re-deriving one), that the LM head is included in vocabulary blocks, the measured BIT-EXACT result on kernels-h4096-b8 with its repeats and edge case, that it is a reference rather than a speed path, and what it does not claim. * docs/npu: the two-arm command sequence for the bit-exact gate A documented reference that nobody can run is a claim, not an instrument: the section now carries the two forward-serve invocations, the dump naming scheme, and the three-step discipline (compare per position, repeat the device arm for determinism, add an edge case). * npu-pack: withdraw the 4096/32/40 row -- a prefix cannot be inferred, only measured against a twin I added this row on granite-4.1-8b because its 1.9 header is clean (40 uniform runs at stride 107,728,192) and its tail is exactly 8,256 + 1,568 x 143,616, which pins the TILE by head-grid arithmetic. That was the mistake: the tile is not the whole rule. `prefix` is defined by the twin diff -- tools/fconst19_measure.py:104 walks `prefix += 1` while the 1.9 block still equals the 1.8 one, so prefix is the first byte where they diverge, and the tile is found by where that first-differing run reappears. Neither number is named anywhere in the 1.9 pack alone. The row's own in-block checks rejected it, which is the argument for removing it rather than for keeping it with a caveat: scale/zero exactness 0 %, second header 0, and 0 of 80 RMSNorm anchors found. A row that fails the pack it was written for is not a measured row. Confirmed against the twin I already had: Qwen3-8B's prefix 12,608 is exactly the first byte where its 1.9 block diverges from its 1.8 block (1.9 `8782 0777...` vs 1.8 `6756 ea78...`, 0 differing bytes in [0, 12608)). amd publishes no 1.8 granite, so granite stays refused by the shape gate -- the honest answer for a layout nothing can establish. The table now says that, so the next person does not re-derive the row from the tail arithmetic the way I did. Qwen3-8B is unaffected: still "all npu-pack checks passed (including the pack leg)". * npu-pack: the h4096 36-layer fconst row, so the Qwen3-8B pack imports at all `kFconst19Rules` held the 32-layer h4096 row and nothing for 36 layers, so `npu-pack info` refused `amd/Qwen3-8B_rai_1.9.0_npu` by design: "its block shape has not been measured for hidden 4096, 32 heads, 36 layers … the 1.8 pack of the same model is what establishes it". Measured now, with tools/fconst19_measure.py against amd/Qwen3-8B_rai_1.8.0_npu_16K: {4096, 32, 36, 12608, 5, 1, 2297856} 26,104 of 3,756,627,264 bytes differ (6.95e-06 of the file) -- content, not layout Every field but the layer count is identical to the 32-layer row (prefix 12,608, five tiles of 2,297,856, rotation 1), which is the first row here where the layer count moves nothing but the number of blocks: the h2048 pair moves both the prefix and the division with it (4,160/9 tiles at 36 layers against 8,512/3 at 24), so this row is measured rather than inferred from that one. The consequence beyond this pack: a kernel family and an fconst rule row are not the same partition. One family can span several rows, so each new pack of a family may cost a measurement before it costs a gate. Verified end to end: the import now completes -- 5,760/5,760 .fconst tiles survive decode->encode byte for byte, 144 attention projections plus 108 MLP projections with fp32 scale/zero exact for 100.00% of groups, 8 full_i8 ELFs linked from kernels-h4096-im12288-b8. * npu-pack test: the new 36-layer row goes in BOTH places, as the guard's own comment requires CI caught the omission and said exactly why: "fconst19 rule: the table has exactly the 11 measured rows (12); a dropped row is a silent un-measure of a whole class, an added one must be added here too". The expected rows are literals copied from the measured table rather than read back from it, so that a wrong digit or a dropped row fails -- which is the arm working, not a nuisance. Added {4096, 32, 36, 12608, 5, 1, 2297856} to it, and the arm now passes. * npu-pack: fused-MLP attributes are not graph matmuls, and the 4096/32/36 row is pinned in the test Two things found by running the tests and real packs on main + this branch. 1. main's row-count arm (#1861) pins kFconst19Rules as literals; the Qwen3-8B row was not in it, so the merged tree failed 11 != 12. Added. 2. The SSMLP/GQO gate/up/down attributes were pushed into OnnxModel::matmuls. matmuls.empty() is how the importer knows a 1.9 pack keeps its head in the .fconst tail, and every 1.9 pack has a fused MLP, so the block-32 tail head (#1948) was no longer read: Qwen-2.5 1.5B 1.9 imported 338 tensors instead of 339, with no lm_head, exit 0 and no refusal. The attributes now go to OnnxModel::fused_mlp, which only intermediate_size_from_graph() reads. New arm 2b builds the tied block-32 fixture with an SSMLP node; with the old push it fails three assertions. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * npu-pack test: arm 2b removes its second fixture directory It was left under the golden directory after a run. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…is the onnx/ component (engine#2000) (#2006) Two doc gaps, both checked against current main rather than taken from the issue. 1. docs/onnx-pack.md: a 1.9 pack's *.onnx component can require vendor op domains (com.ryzenai:DynamicPad, matmul_add_to_sd_gemm, mha_to_sd_mha, conv_to_nhwc_conv) and will not load under the plain ONNX Runtime EPs, so there is no CPU-EP oracle for it -- only the compiled partitioned/ route. Placed as the exception to the existing 'No vendor operators, ever' bullet, which is about OUR exports. 2. docs/onnx.md: the any-graph section now says which binary carries --input, that ort_run lives in the onnx/ component (add_executable in onnx/CMakeLists.txt), that ninja onebit does not build it, and where it lands. Measured here: cmake -B /tmp/bort -DONEBIT_ORT=ON && ninja ort_run -> /tmp/bort/onnx/ort_run, so the page's build/ort_run examples were one directory out and are corrected. Correction to the issue's second claim, which the docs now reflect: '1bit ort-run' does NOT reject --input on current main. app/main.cpp dispatches ort-run to onebit_ort_run_main inside #ifdef ONEBIT_ORT, and onnx/ort_run.cpp handles --input; with ONEBIT_ORT off the subcommand does not exist at all (1bit: unknown command 'ort-run'). So the gotcha is the configure guard, not the flag set. Co-authored-by: agent <agent@local>
…#1990) (#2005) * npu: the attention rows of a prefill block share one thread team (Refs #1990) A batched pass called attention() once per row, and each call opened its own team over the heads: 256 teams per 16-row block on the served 27B pack. The rows of a layer now run their position-local half first and then all (row, head) pairs in one statically scheduled loop. Same statements per pair, so no bit moves: NPU_ATTN_CHECK=1 compares against the per-row form, and attn_math_test holds rows_context() to it on 1, 2, 5 and 12 threads. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * docs: the one-team attention halves its term, but a block-wall gain is not resolved (second ticket) A second ticket (A B A B, quiet box) reproduced the attention term (24.5/26.6 -> 13.0/13.1 ms) and did not reproduce the 17 ms block-wall gain of the first pair (+4 and -24 ms). The section no longer quotes 2.6 %. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…ath, measured (Refs #1785) (#1982) * forward-serve: --prefix-pin, the pinned shared prefix on the device path (Refs #1785, F5) With --prefix-pin the forward keeps the previous request's prompt ids, and when a request shares a prefix with it that no slot covers, cuts the prefill one token short of that point, snapshots and pins it. serve carries the flag through. Off by default. Measured on the NPU: the third question under one system prompt resumes from 341 of 355 tokens, with answers identical to the cache-off run. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * tests: pin_point arms for a shared ending and the common-prefix bound (Refs #1785) Answers the review on #1982: a shared ending under different openings is not a pin point, and a pin point never reaches past the tokens both prompts start with (swept over shared lengths around prefix-min-gain). Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
… (follow-up to #1921) (#1978) #1921 put the refusal on encode_tile(). tile_q4_impl() (tile_q4_1 and tile_q4_1_f32) packs with the same `& 0x0F` and is the writer engine#1918's wrap was measured on. Its cause was fixed at the decoder by #1874, so nothing produces such a code today; this keeps a future caller from having one packed silently. The refusal names the row and column and is made before any byte is written. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…entry (engine#1716, #1956) (#2025) docs/npu.md still said nothing prunes the head images. Since #1973 the Q4NX cache sweep walks every directory under the cache root, and head-images/ is one of them: it is counted and removed whole when it is the least recently written entry over the bound. Say so, and say what its age is (a tile write, not a read). Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…UF converter refuses what it cannot name (Fixes #2013) (#2016) tools/qwen35_to_q4nx.py is the converter the dev box's 27B container names in its own metadata; it lived only in a lane worktree. It now refuses a rotated (prism.hadamard.*) or PTQ1_0 input before writing, and its verify compares the conv1d value-band direction on bf16-rounded sums. tools/gguf_to_q4nx.py lists every tensor kind it has no name for in one refusal (a suffix-less blk.N.ssm_a was a ValueError), names the architecture and the converter that does pack it, refuses an unknown ggml tensor type by number instead of an enum traceback, and leaves no empty output directory. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…orward route; an over-context request is a 400 (Refs #1785) (#2029) * serve: --flash, the single-turn NPU profile; --ctx-size reaches the forward route; an over-context request is a 400 (Refs #1785, F5) --flash = --ctx-size 1024 + --prefix-pin, refusing what contradicts it. serve dropped --ctx-size at the hand-off to forward-serve on a model-generic container; it is carried now and forward-serve lowers its limit to it. A request that does not fit was a 500; it is a 400 with type exceed_context_size_error. Measured on the NPU. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * tests: serve --flash refuses what contradicts the profile, with no model or device (Refs #1785) Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…2024) test_q4nx_container() builds its fixtures under the directory it is given, which is tests/golden when CTest runs it, and never removed them. Every run left tests/golden/q4nx-container/ (container-genre, huge.q4nx, meta.q4nx, short.q4nx, two.q4nx) in the source tree as untracked files. The directory is now cleared on entry and removed on every way out. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
… is visible (engine#2028) (#2041) engine#2028 reported the per-op dump silent on the design path. It was a binary linked before #1977 (cl-npupack-b measured 869/1061 [op] lines on one that has it, and 0 on the filer's, whose strings lack the variable's name), but four runs went to the wrong mechanism first, because "the dump prints nothing" and "this binary has no dump" look identical. ops_dump_announce() prints [ops-dump] NPU_INFER_DUMP_OPS is on: an [op] line follows every projection and weighted RMSNorm, on stderr once per process when the dump is on, from Q4nxNpuForward::init_impl (serve and npu-run, host and device). No line with the variable set means the binary predates the dump. Pinned in ops_dump_test: on prints it once (second call silent), unset and 0 print nothing. On a two-layer container the line comes before the first of 316 [op] lines; with the variable off there is none. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…m it settles (engine#753) (#2037) The issue says zero-copy execution cannot work because XRT cannot bind a foreign dma-buf as a kernel argument. That is no longer true in this tree: npu/npu_rt_api.h declares `ext::bo(const device&, int dma_buf_fd, size_t, void* host)`, the private runtime imports the fd with DRM PRIME and pins those pages, and npu-kernels#39 already runs kernels with one. What was missing was any arm *in this tree*: the shim test covered `ext::bo(dev, size)` only, and the fd constructor appeared solely in a link-coverage function that is never called. Two changes: - `tests/npu_rt_shim_test.cpp`: the fd constructor joins the link-coverage surface, so a runtime library that lacks it now fails to LINK rather than at the first hybrid call site. - `tests/npu_dmabuf_import_test.cpp`: the executing arm. A dma-buf with no GPU in the path (a sealed memfd handed to /dev/udmabuf), imported with the EXPORTER'S mapping, then **bound as kernel argument 0 and run**. Measured under one npu-hold ticket (queue waited 440 s at a 70 C die, command 1 s): ok a dma-buf exists with no GPU in the path (udmabuf over a sealed memfd) ok ext::bo(dev, fd, size, host) accepts it ok the imported BO is the dma-buf's size ok map() IS the exporter's mapping (a second mapping puts the BO on the driver's invalid list and the next submit HMM-faults) ok a write through the exporter's mapping is visible through the imported BO (one buffer, not a copy) ok host = nullptr constructs (so the trap is at SUBMIT, not at construction) (no throw) ok and it is a SECOND mapping, not the exporter's -- the state npu_rt.cpp says the next submit faults on ok the imported BO is accepted as kernel argument 0 (no throw) ok and the run with it completes (COMPLETED) all npu dmabuf-import checks passed The last two lines are the claim: an imported buffer is not merely mappable, it is submit-able -- the binding is where the issue said execution failed. One correction that only the device run could produce: my first version asserted that `host = nullptr` *throws*, because the header's comment reads as if the driver's own mapping is refused at construction. It constructs fine and hands back a second mapping; the documented fault (amdxdna_insert_pages -> amdxdna_mark_mapp_invalid, then an HMM fault on the next submit) is at submit time. The arm now asserts the observable precursor and deliberately does not court the wedge to watch the fault -- a test that wedges the array to observe a documented failure is a bad trade. Skips (77) without a kernel ELF, so a box with none registers a skip rather than a red row. Refs engine#753 Co-authored-by: t <t@e.com>
…he slow route (refs #2106) (#2125) * hw-gate: an npuserve phase, so the gate can fail a serve that is on the slow route (refs #2106) scripts/hw-gate.sh --phases npuserve runs tools/npu_serve_smoke.py on HW_GATE_NPU_SERVE_BIN / HW_GATE_NPU_SERVE_MODEL under the same NPU claim as the npu phase and reports the four assertions in the summary row. PASS needs the tool's exit status and its RESULT line. Not in the default phases; skips with no NPU or no settings. The tool's self-test now also drives its start / ask / stop path against a stand-in serve (fast, slow, dead), and tests/hw_gate_npuserve_test.sh (ctest hw_gate_npuserve) runs the phase end to end against the same stand-in: 13 checks, no device, the fleet's queues untouched. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * hw-gate npuserve tests: hold the loopback-port lock, and write the stand-in owner-only (CI at 414115e) CI's build job failed on route_port_lock: hw_gate_npuserve starts a loopback server without RESOURCE_LOCK route_ports. Both it and npu_serve_smoke_self (which now starts one too) take the lock. CodeQL flagged the stand-in serve's 0755 as world readable; it is written 0700. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * hw-gate npuserve: the serve cannot outlive a TERMed run; the gate passes its own floor, length and reply count (review of #2125) CHANGES at fe89793: when the ticket's hold ends, the TERM reaches the smoke script (a flock sits between them) and Python's default action skipped the finally that stops the serve, leaving 1bit serve running with no ticket. The script now turns TERM/HUP/INT into SystemExit (the serve is stopped, exit 128+signal) and starts the serve with a parent-death signal. Pinned twice: the tool's self-test TERMs a real run mid-request (without the fix: exit -15 and the serve left running), and hw_gate_npuserve_test.sh sends gpu-hold's kill sequence to the phase's flock. READY-with-asks at fe89793: the phase now passes --floor, --max-tokens and --replies itself (30 tok/s, 96, 5), so the rate assertion does not depend on a default in another file. The floor is 30 because single fast-route replies have been measured from 48.8 to 101 tok/s on this shared box and the slow route is 1.9. Docs say the phase is blocking. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…st no longer fails the graph (fixes #1925) (#2134) One fork commit past 7a1f2b81c: - 1703bfeaa (#302): the scheduler's dependency walk no longer counts an in-place CPY's own output as a pending input (a ggml_cast stalled the walk and everything after it was taken in index order, where the L2_NORM root of the fused gated delta net comes before the projections it reads); the fused matcher keeps a match whose layout alias was already passed. Measured on gfx1151 with the two changed objects replaced in an add-on build: Flash-Next UD-IQ3_XXS 18,782-token request, unsupported HRX node 1 -> 0, 390 of 390 fused roots accepted, the whole prefill runs. Qwen3.5-4B old vs new library: 128 of 128 tokens identical in three pairs, the same 33 dispatch registrations, decode 57.4/58.1 vs 58.3/58.2 tok/s. Host: test-hrx-gdn-passed-alias 7 cases, hrx-backend-test 66/66. Not shown: the 18.8k answer with the mask chain on the device against the CPU; those requests end in #1536's all-NaN logits on the old and the new library alike. registry/architectures.json: the recorded source pin only. docs/hrx.md: the #1925 section. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…CPM-V 4.7 pack with the engine's own steps (engine#1788) (#2132) * npu: minicpm_image_probe, one picture and one question through a MiniCPM-V 4.7 pack with every step the engine's own (engine#1788) RGB bytes -> plan_image -> preprocess -> tower_forward (host) -> image_tokens + prompt_positions -> the text model's step_batch_embeds() with the tower's rows where the placeholders are, then a greedy decode. No image decoder (PPM or a generated colour), no tokenizer or template outside the engine. Measured on MiniCPM-V-4.7-1B-NPU2 with the d3-lite kernel set, text model on the NPU (one npu-hold ticket, 99 s): no image -> 'black and white' solid blue -> 'blue' left red, right blue -> '... split vertically down the middle. The left side is red, and the right side is blue.' Host arm, solid red -> 'red'; the prompt's 93 token ids are identical to the reference processor's. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * docs(npu): the image probe needs the padded-head rule (#2026) to load the published packs Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…2111) (#2131) An empty content with finish_reason "length" read like "the model said nothing" when the whole budget went into a thinking block (#2111; it produced #1998 as a wrong bug report). The text was already returned, as reasoning_content, on both routes: what was missing is the count. Both NPU chat routes now report OpenAI's usage.completion_tokens_details.reasoning_tokens, additively: - forward-serve: each generated token's text is kept (not only with logprobs), and the count runs the same AnswerStream that splits the reply (npu::reasoning_token_count, beside content_mask), non-streamed and in the streamed usage chunk. /v1/completions carries no details. - unified route: the generated ids (end token excluded, as completion_tokens excludes it) through this route's ThinkSplit (think_reasoning_tokens); 0 when nothing is split. A token is reasoning when none of its text reached content and the block was open before it, after it, or it wrote into it; leading tokens that produced nothing join the block if the next token opens it. So both tags, a split tag, and an unclosed block at the end of the budget count, while tool-call markup and the layout after a block do not. Tests: npu_answer_parser_test (+10 arms, incl. the issue's 8-of-8 row and a block opened after content) and think_split_test (+6). Mutants of the rule (no "before", no "after", content counted, no pending) each fail an arm. docs/serve.md: what the field means and what to do about it. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…shrug (engine#2077) (#2091) `pr_agent` reads red on PRs because it is CANCELLED, not because it failed: the job carries `concurrency: { group: pr-agent-<pr>, cancel-in-progress: true }`, right for a fast check and wrong for a minutes-long 30B review on one self-hosted runner. Nothing in the tree counted that third state. Measured while writing this (`gh run list --workflow=pr-agent.yml --limit 40`): --event=pull_request cancelled=22 success=17 running=1 -> 55 % cancelled (all events) cancelled=12 success=9 skipped=9 -> 30 % cancelled, the denominator lesson The two differ because `skipped` runs from non-PR events dilute the share, which is why --event exists. `--runs` reads conclusions through `gh run list` (read-only) and reuses this script's contract: verdict line first, counts second, rc 0 PASS / 1 FAIL / 2 nothing readable. The summariser is a pure function over conclusions on stdin, so its arms need no GitHub - including "no runs read is not a pass" and "one cancelled run is already over 0 %", i.e. the threshold's floor is reachable. --max-cancelled-pct defaults to 25. Rebased twice onto a moving main; both conflicts kept BOTH sides: main's six policy_verdict() arms plus my six run arms, one count (22 - counted from the printed arms, not from memory: 20 chk/chkP assertions + 2 case arms). The fix for the cancellation itself is one word in .github/workflows/pr-agent.yml, which no lane token can push; this makes the size of the problem measurable and testable from a lane, and names the cause.
Collaborator
Author
|
VERDICT: NOT READY at Measured, at the object level rather than from PR state: Those Why that matters more than a normal stale branch: this PR's whole purpose is to make What to do, and it is one command plus a re-run: If that last diff still shows anything under What I would keep from this PR: the two paragraphs explaining the conflict and its resolution are exactly what a merge-up should say, and the note that #1981 has five READY verdicts at |
…en only the timing failed (Refs #2106) (#2140) The gate's rate was read three ways on #2106 in one night: 93-94 tok/s, 50 tok/s, and "a function of answer length". It is none of the last two. On the shipped binary the median is 85.7-88.7 tok/s at 16, 32, 96 and 128 answer tokens on a quiet box; the 50 was a loaded box (the route keeps about 1.5 CPU cores of host work per token, and npu-hold serialises the device, not the CPU; I measured 51-89 beside a 10-job build of my own). So the tool now prints one `load` line: the 1-minute load average as the run found it. When the route and the content pass and only the timing fails, the line says a busy box is the likely cause and to re-run on a quiet one. It never changes the result. The load is saved in load.json so --judge reports the run's box, not today's. Self-test: 7 new checks (a pass blames nothing; the busy-box row is named; first token alone counts; the slow route and empty content are never excused; no load average, no line; the result stays FAIL). Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…5) (#2139) Orchestrator decision on #1945 (technical calls delegated by the owner): HOLD. config/llama-mirror-hold.config names the mirror's commit, so the daily run reports the hold, moves nothing and succeeds; the merge of AMD's rebased line is post-release work. A held run also leaves the engine's llama.cpp pin where it is. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…drift while it is being finished (#2114) * ci: the engine#1962 narrowing enumeration is a ratchet, so it cannot drift while it is being finished engine#1962 is a release-blocker (a dense MUL_MAT returns NaN or wrong values when an activation exceeds the f16 range). Its repairs land on the *producers*: a plan records a 2^-6 prescale, the consumer scales back. The issue's remaining question is "which of the narrowing consumers are protected", and the enumeration published on the thread answers it the only way available today - by listing the 25 .loom bodies that narrow f32 to f16 and finding that **none of them carries the guard** (the 5 files that do are producers, and are not in the 25). That answer is worth exactly as long as the list cannot change underneath the person acting on it. tools/ci/hrx_f16_narrowing.txt: the 25 bodies, as a baseline at pin 7a1f2b81c, with the reason the list is a ratchet rather than a bug list written at the top of the file (a body narrowing a bounded scale or a weight is perfectly correct). tools/ci/hrx_f16_narrowing.py: --check fails on a body that appears without a line *and* on a line whose body is gone (a list that can only grow cannot see a rename, engine#1728); --write rewrites the baseline; --self-test covers both arms on fixtures, plus "a '#' line in the baseline is not a path" and "a missing corpus returns 77, not 0". --check skips 77 without the submodule - the corpus is third_party/llama.cpp - so the CI row tests the checker and the arm that needs the corpus runs where it exists. Registered as ctest tool_selftest_hrx_f16_narrowing (runs everywhere) and hrx_f16_narrowing_check (skip 77 without the corpus). Verified on the real corpus: `--check` -> "25 narrowing body(ies), none new, none stale; 0 of them carry a guard marker (the guard sits on the producer, engine#1962)"; adding a probe .loom with one fptrunc -> `NEW narrowing body loom-libs/ops/_ab_probe.loom ... FAIL (1 new, 0 stale of 25)`; removing it -> clean. Refs #1962. * ci: the baseline carries the full notice, and copyright.py learns the file Answers the CHANGES at d38ce8d from two reviewers, both on the same point: the `build` job's "Copyright notice on every file" step fails on tools/ci/hrx_f16_narrowing.txt, because `.txt` has no comment style the checker knows. Both parts are needed and both are here: 1. the baseline carries the same full Apache header as tools/ci/add_test_gaps.txt (the other '#'-comment data files) instead of a two-line stub; 2. `hrx_f16_narrowing.txt` joins the '#' entry in LINE_STYLES next to `add_test_gaps.txt` / `unreferenced_*.txt`, which is the same kind of file: data whose header IS the notice it would otherwise be asked for. Verified: tools/copyright.py --check rc 0; the checker still reports "25 narrowing body(ies), none new, none stale" against the corpus at the engine pin (checked with the corpus symlinked in, then the link removed); --self-test 7 arms ok. Note on why I did not see this before pushing: I ran `copyright.py --check` while the file was still untracked, and the check does not see untracked files. That is the hole I reviewed earlier tonight in the other direction (copyright_check_untracked_test.py); this is the same hole from the inside. Refs #1962. --------- Co-authored-by: bong-water-water-bong <bong@water>
…ugh a symlink (CodeQL world-writable alerts; unblocks #1830) (#2057) * npu: dumps are created 0600 and the head-image cache 0644, never through a symlink (CodeQL world-writable alerts) Six open CodeQL cpp/world-writable-file-creation alerts on main, all fopen(..., "wb"), which creates 0666 & ~umask and follows a symlink planted at the name: npu/forward/src/q4nx_forward.cpp NPU_I8_GS_DUMP host-arm dump npu/target/xdna/xdna_target.cpp NPU_I8_GS_DUMP device dump npu/forward/src/q4nx_head_image.c head-image cache temp file npu/forward/include/residual-dump.h NPU_INFER_DUMP_DIR layer dump tests/npu_pack_import_test.cpp, tests/npu_residual_dump_test.cpp A PR that merely touches nearby code inherits these as "new alerts in code changed by this pull request": #1830 has sat on a CHANGES verdict for one since 14:35. npu/forward/include/file_create.h (plain C, shared by the .c and .cpp sources): onebit_fopen_private_wb (0600) and onebit_fopen_shared_wb (0644), open(O_CREAT|O_TRUNC|O_NOFOLLOW|O_CLOEXEC) with a constant mode, then fdopen, with errno kept from the failing call so the residual dump's strerror message is unchanged. The test's /dev/full probe opens for reading. Measured with umask 0: 600 and 644; a symlink at the path is refused (ELOOP). ctest npu_residual_dump, ops_dump, npu_design_grid (the head-image cache arms) and npu_pack_import pass; onebit_npu_forward builds in the ONEBIT_NPU configure. The path-injection alerts at the same sites are left alone: those paths come from env variables by design, so they are the owner's call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * file_create: narrow an existing file's mode, pin it in a test, and refuse new fopen-for-write sites (review on #2057) Review asks at 2ecd5fd: - O_CREAT's mode applies only on creation, so a dump an older build left at 0666 stayed 0666. The opener now clears every bit beyond the stated mode with fchmod on a regular file. It only narrows: a 077 umask still leaves the cache 0600, and a device (/dev/null, /dev/full) keeps its own mode. - tests/file_create_test.c (plain C, umask 0): new 0600/0644; an existing 0666 narrowed and truncated; 077 not widened; a symlink refused with ELOOP and its target untouched; ENOENT kept for strerror; /dev/null not chmod-ed; the append opener. Against a header that reverts to fopen it fails 9 of 14. - tools/ci/file_create_check.py (--check / --self-test, both ctest rows): refuses fopen(..., "w"/"a") in tracked C/C++ unless the line says `file-create-ok: <reason>`. Its first run found three sites my grep had missed (the call's c_str() parens): forward_serve's logits dump and ort_run's output dumps, now through the openers (with a new onebit_fopen_private_ab for outputs.txt). The /proc write in onebit_ep_qdq_proof.cpp and the Windows fallbacks carry the marker. - The header names who reads the 0644 file (the head-image cache, read back by a later serve), so the shared opener is not reused for a dump. ctest file_create, file_create_check(_self), npu_residual_dump, ops_dump, npu_design_grid, npu_pack_import pass. In the ONEBIT_NPU configure, forward_serve.cpp compiles and 1bit links. ort_run.cpp passes -fsyntax-only against the box's ONNX Runtime headers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * tests/file_create_test: start the narrowing arms at 0664, not 0666 (CodeQL on #2057) The test chmod-ed its fixtures to 0666 and created the symlink target with fopen, and CodeQL flagged all four lines as world-writable file creation, which is fair: the test should not make the files it exists to prevent. 0664 (what fopen left under a 002 umask) is wider than both openers' modes, so the arms still check the narrowing; the symlink target goes through the private opener. ALL PASS; against a header reverted to fopen it still fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…(Refs #1786) (#2144) FastFlowLM's Phi4-mini-Instruct-NPU2 names <|endoftext|> in config.json and tokenizer_config.json, and its template closes every turn with <|end|>. The serve stopped only on the first, so the answer was 'Paris<|end|><|assistant|>...' to the token budget. The marker is now taken from a rendered two-message exchange (npu::turn_end_marker). Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…e#2129) (#2141) A build compiles whatever is in the working tree. Every check in tools/ci reads the tree it is handed, so none of them can see that a submodule is checked out on a different commit than the superproject pins - a build that compiles an un-pinned dependency passes every gate we have. Measured on 2026-11-11, which is why this exists. The shared ~/1bit-engine had FOUR submodules off their pins: third_party/llama.cpp c16424f5620d (pinned 7a1f2b81ce17) third_party/hrx-system 51b1739ae5fd third_party/tokenizers 88a4498ad4ea third_party/xdna-driver 5d302c9986a5 and the engine compiles common/jinja from llama.cpp (CMakeLists.txt:147). That checkout was BEHIND the pin on the files it compiles: behind f08bd76f6 "jinja: accept \xNN escapes in the lexer (#135)" plus a std::to_string portability fix. Jinja decides prompt rendering, so an engine built there renders a \xNN-bearing template differently - not a build failure, but a difference that reads as a model bug. `git submodule status` marks it in one character, and this is the check for it: ' ' at the pinned commit -> pass '+' a different commit (the #2129 shape) -> FAIL, naming both shas and the two repairs 'U' merge conflicts in the submodule -> FAIL '-' not initialised -> skipped by default, FAIL with --require-initialised The default compares only what is checked out, because a configure that never inits a submodule is not this check's business; --require-initialised is the release-gate form, where a missing checkout is a failure too. --check on this branch's worktree: 7 submodules, 0 at their pin because a fresh worktree inits none, rc 0 with the "./" default and rc 1 with --require-initialised. Against the shared ~/1bit-engine: rc 1, naming exactly the four above. Verified: --self-test 16 arms ok (each flag, the three failure shapes, "no submodules read is 'cannot tell', not a pass", and "one '+' among three lines still fails" so the check is not first-line-only); registered through cmake/tests-*.cmake so it needs no ci.yml line - ctest -N lists submodule_pin_check and _self, and both pass.
Collaborator
Author
|
CORRECTION to my verdict above, and then the real defect — which is worse for this PR than the one I originally gave. I was wrong that a stale base would "revert main's files". The real defect is inside the file the resolution touched, and it is a row loss: So after this merges into The right acceptance check, and it is a set comparison, not a diff: comm -23 \
<(git show origin/main:CMakeLists.txt | grep -oE 'add_python_test\([a-zA-Z0-9_]+' | sort -u) \
<(cat <(git show <head>:CMakeLists.txt) <(git show <head>:cmake/tests-python-tools.cmake) \
| grep -oE 'add_python_test\([a-zA-Z0-9_]+' | sort -u)
# must print nothingFor reference, on this lane's other PRs the same check gives: #2130 missing Verdict stands: NOT READY at |
…roduct: 37.7 -> 3.2 s per 448x448 view, same bits (engine#1788) (#2142) tower_forward() ran every loop on one thread and walked each weight matrix once per token, so a 448x448 view (1,024 patches) of the 1B pack took 37.7 s. The row and head loops are now spread over threads (ONEBIT_VISION_THREADS, vision::tower_threading) and linear() applies each weight row to 16 tokens before moving on. Every token and head is still computed by one thread with the same sequence of operations, so the output is the same bits. Measured, box load ~13: 37.7 s before; 18.1 s on one thread (the blocked product); 3.2 s on 16 threads. Test: the fixture's goldens hold, and 1, 2, 5 and 16 threads (forced, no size threshold) give byte-identical rows. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…ed rows (Refs #1786) (#2148) Eleven packs with a row from the dev box: six on the fused route with nothing edited, six on the per-op route (one pack on both), one decision pack; the same-pack pair against FLM's runtime; what each route needs; the open fault on the fused route; and what the rows do not establish. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
… ones (#2027) * tools/ci: a /tmp literal is RAM on this box, so the ratchet stops new ones `/tmp` is tmpfs on both boxes, so a debug dump, a staging copy or a benchmark log written there is anonymous memory: it cannot be reclaimed to disk under pressure, it goes to swap, and when swap fills every lane on the box stalls. Measured 2026-11-10 21:15Z: /tmp held 52 G of 62 G and swap was 71/71 GiB full; the two largest items were a stale 1bit-fconst19*.fconst scratch copy (3.5 G) and five q17-*.json dumps (9.2 G). engine#1907/#1952 bounded the fconst scratch. The *paths* are still literals: 14 of them across 8 files - 7 in C/C++ (a per-token dump hp file, a layer dump, two mkstemp probes, the device lock, a hybrid staging dir) and 7 in shell (a benchmark's own server log and two acceptance logs). The per-token one, q4nx_hidden_%d.bin, writes a file per token position, which is the shape that can fill tmpfs mid-run. This adds the check, not the fix: tools/ci/tmp_paths.txt baselines the 14 sites and may only shrink, and a new one fails. Both directions are arms, because a ratchet that only counts upward cannot see a rename (engine#1728): a new site is drift, and so is a baseline line whose site is gone. Verified: --check on this tree reports "14 baselined, none new, none stale"; --self-test's 9 arms pass, including "a commented-out path is not a site" (the engine#1983 substring trap), "/var/tmp is not /tmp", and "a renamed file leaves a stale baseline line". Registered through the cmake/tests-*.cmake glob (engine#1979), so it needs no CMakeLists.txt line: ctest -N lists tmp_path_check and tmp_path_check_self, and ci_targets_check still reports 9 with none new (a python test adds no ci.yml --target). The rewrite of the 14 sites wants the resolve order ONEBIT_DUMP_DIR -> $XDG_CACHE_HOME/1bit/scratch -> $HOME/.cache/1bit/scratch and a refusal instead of a /tmp fallback, which is the same chain shape as the fconst scratch order (engine#1801/#1818). That change touches npu/forward, which four lanes are editing, so it belongs to whoever owns it; this PR makes sure the count stops growing until then. * tools/copyright: list tmp_paths.txt by name, beside the two other #-comment baselines refused the new baseline: `.txt` has no comment style of its own, and a blanket "*.txt" would demand a notice in files that are data. The two existing ratchet baselines (add_test_gaps.txt, unreferenced_tests.txt) are handled exactly this way, so this lists the third beside them - and --fix then wrote the notice it wanted, which is at the top of the file. * tmp_path_check: match the BARE spelling too - the review measured a 14-site baseline that was a 45-site tree The reviewer's hole, reproduced before fixing: echo x > "/tmp/quoted.log" -> caught echo y > /tmp/bare.log -> INVISIBLE The pattern required a leading quote, so `> /tmp/x.log`, `cp a /tmp/x` - the plain shell idiom, and the most likely form in scripts/*.sh - never matched. A ratchet that cannot see the common spelling of the thing it guards is a ratchet over quoted literals only, which is not what the 21:15Z directive was about. The anchor is now "not the tail of another path" instead of "a quote": (?<![A-Za-z0-9_./-])/tmp/[^"'\s;|)]*["']? `/var/tmp/x` stays out (the `a` before `tmp` blocks it) and `/tmp/x` is caught after a space, `>`, `=`, a quote, or nothing. A trailing quote is stripped, so the baseline records the PATH: re-quoting a site is not a new site, and one path spelled two ways is not two lines. Re-measured after the fix, and the old baseline was an undercount by a factor of three: before 14 sites in 8 files after 45 sites in 18 files Ten files were entirely invisible: scripts/check-ubuntu-2604-lemond.sh (6 sites, including a downloaded `/tmp/engine.tar.zst` and its extracted tree), scripts/pkg-proof.sh (5), tools/npu-arena/*.sh (4), tools/showcase-612*.sh (2), tools/xhci_boot_capture.sh, tools/repro-hostgemm-lock.sh. Two new self-test arms assert it: "a bare /tmp path is a site (no quotes)" and "the bare form is recorded without a quote it never had". Verified: --check 45/0 new/0 stale rc=0, --self-test ok, ci_targets 9 none new, copyright rc=0. Also note the reviewer's second point: the red build at 110ba63 was `dist.tar.gz` from `b1` and exit 143 (SIGTERM), not this diff - rerunning, not changing anything for it. * tmp_path_check: drop the xhci_boot_capture.sh site - main fixed it, and the ratchet's stale arm said so Rebasing onto main made the checker report its own first real fix: tmp_path_check: baseline line has no site any more (fixed or renamed): tools/xhci_boot_capture.sh /tmp/xhci-capture-$stamp - delete it if fixed Verified in main's file before deleting the line: the capture now goes to ${XDG_CACHE_HOME:-$HOME/.cache}/1bit/captures with a comment that says why ("NEVER /tmp: on strixhalo /tmp is tmpfs"), which is the rule this ratchet encodes. 44 baselined sites, none new, none stale, rc=0. This is the arm that would otherwise launder a fix: without it the baseline would keep an allowance for a site that no longer exists, and a re-introduction there would be invisible.
…owing pair move into the family file The conflict was the registration block again. The family file now holds main's block verbatim (the 52 lines removed from CMakeLists.txt are byte-identical to its body), and CMakeLists.txt keeps only the include(). Set compare of every add_test/add_python_test name, main vs this head: no difference either way. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
ready for review at d5dc461 This merges current main (
|
|
Persistent review updated to latest commit d5dc461 |
bong-water-water-bong
left a comment
bong-water-water-bong
left a comment
Collaborator
Author
There was a problem hiding this comment.
VERDICT at d5dc461: READY
Re-verified after the newest merge of main (which carried #2027's /tmp ratchet). Still a merge PR onto #1981's branch, and still the right shape: five READY verdicts that cannot land because the tree moved is a cheaper problem to solve with a scoped merge than with five re-reviews.
#1981 is still open, so the obligation I noted on the previous head stands: merging this moves #1981's head, and #1981 then needs verdicts at the new head before it can reach main. That is the third two-PR chain today (#2035→#1931, and now this) where the second half has no named owner — worth someone deciding, because a chain with an unowned link is how a finished fix strands.
bong-water-water-bong
left a comment
bong-water-water-bong
left a comment
Collaborator
Author
There was a problem hiding this comment.
VERDICT at d5dc461: READY
Reviewed head d5dc461 (refs #1979). A merge of main onto #1981's branch fix/1979-python-tests-family. This is the first verdict at this head. The CHANGES at 0140a02 was that the proof had gone stale as main moved, so I checked against main as it is now (11164a5e).
- Static, by name: the
add_python_test(...)names acrossCMakeLists.txtandcmake/*.cmakeare identical at this head and at current main, 34 each. None is registered twice. - Configured: fresh configures of current main and of this head (
cmake -G Ninja, thenctest -N) give 230 and 229 tests. The only difference isnpu_pad_head_rows, which main added in11164a5e, after this head's merge of main ("head predates 11164a5"). In the tree thatgit merge-tree origin/main d5dc461produces,NAME npu_pad_head_rowsis registered once, and no python test is registered twice. So nothing is lost or doubled on the way to main. git merge-treeonto current main: no conflicts. CI at this head:build,hrx_buildand the pin checks green.
Main moves fast here: if #1981 is not merged soon after this, the same comparison is worth one rerun at #1981's merge, since a new row added to main's old python block would land outside the family file again.
Collaborator
Author
|
VERDICT: READY at comm -23 \
<(git show origin/main:CMakeLists.txt | grep -oE 'add_python_test\([a-zA-Z0-9_]+' | sort -u) \
<(cat <(git show <head>:CMakeLists.txt) <(git show <head>:cmake/tests-python-tools.cmake) \
| grep -oE 'add_python_test\([a-zA-Z0-9_]+' | sort -u)
# at d5dc46120: no output (at 0140a027: laya_finetune, logits_kld_self)Registration counts at this head: 34 across the two files against main's 32 — the branch carries main's set plus its own, which is the shape #1981 is supposed to have. And my earlier reasoning stays retracted, so nobody inherits it: a Worth carrying into the PR body if it is cheap: that four-line |
bong-water-water-bong
added a commit
that referenced
this pull request
Nov 11, 2026
…ges teach the instrument (engine#2143) (#2162) #2143 measured it: with `GGML_HRX_LOG_DISPATCH=1` set, an 18,782-token request *completes* where the same binary, model and request fails without it, and NaN-logit incidents go from 2 to 0. It also carries a published rate that is a 2.3x casualty of this: one plain 18.6k prefill measured 271.75 t/s with the flag unset and 118.86 t/s with it set. Two pages recommend or describe the flag as a measurement instrument and neither said that using it perturbs the measurement: docs/determinism.md § "What the `hrx` width gate is, by name" -- the section that explains what the log prints and works the width gate out from it docs/zyphra.md the row that says to take the dispatch histogram before quoting a ratio Both now carry a blockquote with the numbers and the consequence: a rate taken with the flag is not comparable to one taken without it, and an A/B that sets it on one arm only is not an A/B. docs/hrx.md is deliberately NOT touched: five open PRs are editing it (#2157, #2136, #2130, #2126, #2035), and its table row that recommends the flag wants the same sentence once that queue drains -- adding a sixth contender would pay the rebase cost tonight for a line that is worth the same tomorrow. Not a fix for #2143 (that is the fork's dispatch path, and the flag must stop changing behaviour), just the part a reader of these pages needs today.
Collaborator
Author
|
Closing under the lane cap (2 open PRs, orchestrator 11-11 16:45Z). This only merged main into #1981's branch, and the merge queue lands into main only. The branch |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Onto #1981's branch (
fix/1979-python-tests-family), not onto main. #1981 has 5 READY verdicts at61e39b2, but it now conflicts withmaininCMakeLists.txt, the file it exists to stop conflicting. This merge ofmainresolves the conflict. Merging it into that branch makes #1981 mergeable again. Refs #1979.The conflict, and the resolution
add_python_test(...)block withinclude(cmake/tests-python-tools.cmake).mainhas added five rows to that block, which cmake: the Python tool self-tests get their own registration file (engine#1979, first family) #1981's family file does not have:moe_expert_chunks_accum_test(tests: register moe_expert_chunks_accum_test and shrink the unreferenced baseline to 1 (Refs #2018) #2105),qwen35_to_q4nxandtool_selftest_qwen35_to_q4nx(npu-pack: a qwen35 GGUF has an in-tree converter, and gguf_to_q4nx refuses what it cannot name by name (Fixes #2013) #2016),tool_selftest_amdxdna_tdr_report(amdxdna: a kernel-log report of what TDR recovery did, and a test-only patch that makes a restart fail (Refs #1750) #2032), andtool_selftest_dflash_draft_info.include()and replaces the family file's rows with main's block verbatim, comments included, below the file's own header. Main's block is a strict superset of the old family file: 0 rows only in the family file, 5 only in main.Proof
There are no conflict markers left. The resolution touches only
CMakeLists.txt(theinclude()line) andcmake/tests-python-tools.cmake. Everything else in this merge commit ismain.🤖 Generated with Claude Code