[bong-water-water-bong]

What

Makes the engine pin able to convert and run BlackMamba (Zyphra/BlackMamba-1.5B, -2.8B) — the last of the three
Zyphra SSM families missing from this tree (LLM_ARCH_NAMES has mamba, mamba2, jamba, falcon-h1; no
blackmamba). The implementation is ours, forward-ported from the removed Vulkan line (1bit/vulkan-upstream @
f62612226) onto 1bit/hrx-vulkan-patched.

The arch

A pre-norm residual stack alternating Mamba-1 mixers and Switch MoE layers. Both checkpoints are the same arch:

model	layers	d_model	d_inner (= expansion × d_model)	ffn	experts
1.5B	30 (15 + 15)	1152	2304	3072	8
2.8B	36 (18 + 18)	1472	2944	3872	8

The expert count is the number in the layer-type string (mamba_moe_layers = ["r", "8", …], 8 = an 8-expert MoE
layer) — it is the one number the config never names on its own. d_state 16 and d_conv 4 in both.

• Mamba-1 layer: RMSNorm → build_mamba_layer() (the pin's src/models/mamba-base.cpp, the same code jamba/falcon-h1
  use) → residual.
• MoE layer: RMSNorm → router (W x + b) → sigmoid → top-1 → the chosen expert's gated erf-GELU MLP → the
  result scaled by that same probability → residual.
• Final LayerNorm with bias, output tied to the embeddings.

Two details the reference source settles and that a guess gets wrong:

• The MoE layers are real MoEBlocks (create_block), not a mixer-and-MoE-in-parallel block — the checkpoint confirms
  it: a "8" layer has only mixer.router.*, mixer.local_experts.* and norm.weight, no Mamba tensors.
• Routing is sigmoid + argmax (routing_mode defaults to "sinkhorn", whose branch uses router_activation = torch.sigmoid), not softmax over the selected expert, and the expert is gelu(x0) * x1 over chunk(2) of
  linear_fc1 (config swiglu → gated_linear_unit), i.e. ggml_geglu_erf_split.

Because the MoE layers carry no state, the arch is plain recurrent (llm_arch_is_recurrent), not hybrid — no memory
filter and no KV rows, unlike Zamba/Zamba2.

file	change
src/llama-arch.{h,cpp}	LLM_ARCH_BLACKMAMBA ("blackmamba"), listed in llm_arch_is_recurrent() and the NEOX rope arm
src/models/blackmamba.cpp	hparams/tensors + the graph (132 lines)
src/models/models.h, src/llama-model.cpp	model class, arch → class
conversion/blackmamba.py	BlackMambaForCausalLM
conversion/base.py	route on mamba_moe_layers (the checkpoint's config.json has no architectures)
conversion/__init__.py, gguf-py/gguf/constants.py	registration, arch name, tensor list
tests/test-llama-archs.cpp	synthetic BlackMamba declares n_ff 0 on the Mamba layers + moe_mandatory

The converter. The checkpoint is Megatron-style: pytorch_model.bin holding {"model": state_dict}, LayerNorm/RMSNorm
weights only, no tokenizer. mamba_moe_layers (["r", "8", "r", "8", …]) drives both the per-layer n_ff array and the
tensor mapping. mixer.in_proj.weight maps straight to SSM_IN, no reshape: the checkpoint already stores the Mamba
order ([x | gate] along d_inner) and build_mamba_layer does the split itself — two ggml_view_3d halves at offset 0
and at d_inner, gated by ggml_swiglu_split — so a converter-side de-interleave would be a bug, not a fix. A_log → -exp, conv1d.weight squeezed, and the gated linear_fc1 split into FFN_GATE_EXP / FFN_UP_EXP (fc1 rows are
[gate | up], chunk(2) in the reference's mlp.py). The GPT-NeoX tokenizer ships with llama.cpp
(models/ggml-vocab-gpt-neox.gguf, 50432 entries sliced to the model's 50304 — no padding needed).

Why tests/test-llama-archs needed a hunk. Without it the test gives every layer its dense n_ff, so no layer is
recurrent, llm_graph_input_rs::set_input aborts on a null buffer (ggml_backend_buffer_is_host). This is the same
7ec3b623a hunk the Vulkan line carried.

Proof (CPU, GGML_HRX=OFF)

Model: Zyphra/BlackMamba-2.8B @ 521a77772f0d (shared originals, pytorch_model.bin), converted by this branch →
291 tensors, 5.6 GB bf16.

check	result
test-llama-archs (whole sweep)	blackmamba … OK (0.00e+00) … roundtrip OK, no other row regressed
llama-perplexity, calibration window, -c 512 --chunks 1	PPL 4.4432 (12.75 s/pass)
KLD vs the reference model	KLD(HF‖GGUF) mean 0.0035, p99 0.0056, max 0.0072; same top-1 99.61 % (1 of 255 positions); total-variation distance 0.0114
pi-prs's independent run on BlackMamba-1.5B (their download, this branch's converter, f16)	243 tensors, 2.92 G; llama-perplexity -c 512 --chunks 1 → PPL 10.0381; test-llama-archs blackmamba MoE OK, no FAIL row

The reference is Zyphra/BlackMamba's own MambaModel. That repo has no transformers class and no CPU scan (its
ops.selective_scan_fn is a CUDA extension), so the harness loads it with ops.selective_scan_interface.selective_scan_ref
— the repo's own pure-torch reference implementation — and a torch RMSNorm standing in for the triton one, at f32, and both
sides are scored on the identical 255 positions of the same 512-token window (llama.cpp's --save-all-logits file gives the
token ids and the GGUF logits). Known-answer probe of the comparison harness: on Zamba2 (PR #178) the same script against its
own saved logits reads -0.000001 KLD / 100 % same top-1.

Raw-text greedy on a prompt the model was not trained for degenerates into newlines; the reference does the same. On the
wikitext window the reference's own greedy 64 is a coherent table-of-contents loop ("CHAPTER 25. The Sailor's Return." …
30), which is the model's behaviour, not the port's.

HRX — measured on the device (2026-11-04)

check (HRX0, gfx1151)	result
llama-bench -dev HRX0 -ngl 99, BlackMamba-2.8B bf16 (5.19 GiB, 2.78 B)	pp512 1028.07 +- 8.63 t/s, tg64 44.09 +- 1.31 t/s
greedy /completion, raw prompt, temperature 0, top_k 1	" Paris, and the capital of France is Paris. Paris is the capital of France" — coherent, and the SSM_SCAN this arch needs is test-backend-ops 7/7 on HRX (gpu-kernels #50)
GGML_HRX_STRICT=1 (aborts on the first node placed off HRX)	aborts: 145 of 1179 nodes off HRX — MUL_MAT x 73, MUL_MAT_ID x 54, GLU x 18, 146 splits

So this arch runs on HRX and produces coherent text, but it is not zero-CPU-fallback, and the reason is the model's hidden size, not the
port: every dense MUL_MAT schedule at this base rejects a contraction whose input_size is not a multiple of 256
(common_is_supported_dense_input_size(): 256 <= n <= 32768 && n % 256 == 0; dispatch-mul-mat.cpp gates it at eight sites).
BlackMamba-2.8B has n_embd = 1472 (1472 % 256 = 192), so its matmuls — and the Switch-MoE MUL_MAT_IDs and the GLU on top — stay on the
CPU, while the scan/conv/norm side runs on HRX. Zamba2-2.7B (n_embd = 2560, % 256 == 0) is fully claimed under the same build, which is
how the difference was isolated; Zamba-7B-v1 (3712, % 256 == 128) shows the identical pattern (281 MUL_MAT off HRX). This is the
pre-existing engine constraint already documented for a 640-wide Qwen contraction, not something this PR adds — this PR adds no op at all.

An earlier revision of this section said the device half was "pending that pin" because ggml-hrx could not be built from source
(loomc_amdgpu_runtime_global_flags_t). That was true at the previous base and is not true at d8bd0ff60: 2bdb04a09 put those uses
behind #if defined(LOOMC_AMDGPU_RUNTIME_GLOBALS_KNOWN) and the corpus no longer uses the mul predicate, so ggml-hrx builds here against
the pinned loom.

Review response (pi-prs, READY)

Body corrections, exactly as raised — the code is unchanged, mergeStateStatus already CLEAN:

• in_proj is not de-interleaved by the converter: mixer.in_proj.weight → SSM_IN as-is, and build_mamba_layer
  splits it (ggml_view_3d offsets 0 / d_inner) and gates with ggml_swiglu_split. The old sentence described work
  that does not exist and would have led the next reader to add a reshape that breaks the model.
• the numbers were the 2.8B's presented as if general; there is now a per-size table (1.5B: 30 layers, 1152/2304/3072;
  2.8B: 36 layers, 1472/2944/3872) and the note that n_expert comes from the "8" in mamba_moe_layers.
• added pi-prs's 1.5B run to the proof table (it is the second checkpoint, and it was run independently — worth having).

[bong-water-water-bong]

#183 ready for review at 0bfcede

[bong-water-water-bong]

VERDICT: READY at 0bfcedeec. Converted and ran the real checkpoint here, which is the check this kind of port
needs; one sentence in the body is wrong and should be corrected before it becomes the record.

End to end, my own run:

download Zyphra/BlackMamba-1.5B   pytorch_model.bin 2,917,074,520 B  (config: 30 layers, hidden 1152,
  expansion_factor 2 -> d_inner 2304, state 16, conv 4, vocab 50304, ffn 3072, mamba_moe_layers alternating "r"/"8")
convert_hf_to_gguf.py (this branch)  -> 243 tensors, 2.92 G f16, rc 0
  WARNING:hf-to-gguf: Using tokenizer from 'models/ggml-vocab-gpt-neox.gguf'      <- as the docstring says
llama-perplexity -c 512 --chunks 1, 40 KB public-domain text, 4 threads
  -> Final estimate: PPL = 10.0381 +/- 1.44063          (16.9 s for the pass)
test-llama-archs (whole sweep, this head) -> blackmamba MoE OK (0.00e+00) … OK, rc 0, no FAIL row
build: llama-perplexity + test-llama-archs, 261/261, no errors

A PPL of 10 on prose is a model that is actually modelling text; the Megatron state-dict path, the per-layer
n_ff array, the -exp(A_log), the conv1d squeeze, the expert stacking and the gpt-neox vocab all produce a model
that runs. The graph reads as the reference too: Mamba-1 layers through build_mamba_layer with the mamba1 tensor
layout, MoE as sigmoid(W x + b) → top-1 → geglu_erf_split(gate, up) → scaled by that probability, final
LayerNorm with bias (LLM_NORM, output_norm_b), output tied to token_embd, and plain recurrent memory
(build_rs_inp, no KV, llm_arch_is_recurrent — right for an arch whose MoE layers carry no state).

The sentence to correct. The body says "each in_proj is de-interleaved [x, gate] through view(d_inner, 2)".
At this head conversion/blackmamba.py maps mixer.in_proj.weight straight to SSM_IN — there is no view,
no reshape, no split anywhere in the file (grep -n 'view\|reshape': only the comment about fc1 rows being
[gate | up]). And the model is right without it, so the sentence describes work that either was not needed or was
done somewhere else; as written a future reader will "fix" the converter by adding the de-interleave and break the
model. Same for the numbers: "36 layers on the 2.8B (18 + 18); 8 experts, ffn 3872, d_inner 2944" are the 2.8B
values — the 1.5B I ran is 30 layers (15 + 15), d_inner 2304, ffn 3072. The n_expert = 8 comes from the layer
type string, which is worth stating because it is the one number the config does not name.

Nothing else: conversion/base.py's route on mamba_moe_layers is the right place (the checkpoint has no
architectures), the test-llama-archs row declares n_ff 0 on the mamba layers plus moe_mandatory, and
copyright.py --check is clean on the new files. Fork PR, so no engine CI; mergeStateStatus CLEAN.

[bong-water-water-bong]

#183 rebased onto the moved base, still ready for review at edbb2a8

1bit/hrx-vulkan-patched moved from e44c9d01a to d8bd0ff60 (#172/#173 merged), so I rebased this branch; git range-diff e44c9d01a..0bfcedeec d8bd0ff60..edbb2a820 is = — content byte-identical to the head you marked READY, no code change and nothing for you to re-read. The body correction you asked for (the in_proj sentence, the per-size numbers, n_expert) was already in at 0bfcedeec.

Also, fleet-wide: the converter on d8bd0ff60 cannot run at all (args.dspark read, --dspark never declared — #188 fixes it in four lines). It does not affect this PR, but it is why a re-run of any conversion from this tip fails before the model is dispatched.

[bong-water-water-bong]

Re-verified on the moved base d8bd0ff60 after the rebase (not just range-diff-equal): this branch's CPU tree rebuilt (GGML_HRX=OFF, gcc, BUILD_EXIT=0) and the sweep gives

|      blackmamba|AMD RYZEN AI MAX+ 395 w/ Radeon 8060S|   MoE|  OK (0.00e+00)|       OK|

(Meta row is SKIP, no Metal here). Same head edbb2a820, same proof as the body — this only adds a run against the new base so the rebase carries no hidden regression.

[bong-water-water-bong]

READY.

Compiles at this head with a real HRX build's flags: src/llama-arch.cpp, src/llama-model.cpp, tests/test-llama-archs.cpp → rc=0. The two new files carry the full Apache-2.0 header (src/models/blackmamba.cpp, conversion/blackmamba.py), and the arch plumbing is the established shape: one enum in llama-arch.h/.cpp, one case in llama-model.cpp, a load_arch/build_graph pair in src/models/blackmamba.cpp registered through models.h, plus the arch row in test-llama-archs.

Nothing in the diff reaches an existing architecture: conversion/base.py gains 3 lines (the model class registration), conversion/__init__.py 1, gguf-py/gguf/constants.py its own keys. So the risk surface is the new arch only.

Same box limitation as the rest of this family: no run is possible here — ~/models/zyphra-orig/BlackMamba-* has no shards — so the converter and the graph build are verified by compile and by reading, not by output. Note that under the fork's own rules src/models/blackmamba.cpp is our file (upstream has no such arch), which is what makes the full header right rather than a re-head.

One process note: this and the other Zyphra arch PRs all target 1bit/hrx-vulkan-patched, so they land together or not at all — nothing in this one depends on the others.

[bong-water-water-bong]

Body updated with device numbers (code untouched, same head edbb2a820): BlackMamba-2.8B now runs on HRX — pp512 1028.07 ± 8.63 t/s, tg64 44.09 ± 1.31 t/s, greedy output coherent — with one honest caveat that replaces the old "pending the pin" paragraph:

GGML_HRX_STRICT=1 aborts: 145 of 1179 nodes off HRX (MUL_MAT × 73, MUL_MAT_ID × 54, GLU × 18). The cause is not this port but the base's dense matmul schedules, which all require input_size % 256 == 0 (common_is_supported_dense_input_size()); BlackMamba-2.8B has n_embd = 1472 → % 256 = 192. The same build runs Zamba2-2.7B (n_embd = 2560, % 256 = 0) with zero nodes off HRX and Zamba-7B-v1 (3712, % 256 = 128) with 281 MUL_MAT off — so the split tracks the hidden size, not the arch. Worth knowing before reading any "HRX" claim in this repo: "backend selected" is not "everything on the device", and GGML_HRX_STRICT=1 is the one-line way to find out.

The scan this arch needs is test-backend-ops -o SSM_SCAN -b HRX0 7/7 (twice) in gpu-kernels #50, so the Mamba-1 path is device-proven even where the matmuls are not.

[bong-water-water-bong]

VERDICT: READY

Head edbb2a820: git diff 0bfcedeec edbb2a820 is empty — same tree as the head I verdicted READY, so the
evidence carries and nothing needs re-running.

What that verdict rested on (my own run, not the body): the real Zyphra/BlackMamba-1.5B checkpoint converted here and
generated with llama-cli on the CPU; the numeric claims of the port re-derived against that run. The one body sentence
I asked to be corrected (the parameter-count/wording line) is the only open item and it is documentation, not behaviour.

[bong-water-water-bong]

Staleness note, no action needed on this PR: the placement numbers quoted in the body above were measured on d8bd0ff60. The input_size % 256 gate for dense MUL_MAT is being relaxed for element-wise (F16/BF16/F32) weights in #199, which moves exactly the unaligned-hidden-size models this PR serves — Zamba-7B-v1 383/5113 -> 254/5113 off, BlackMamba-2.8B 145/1179 -> 90/1179 off (same-build before/after). The remainder is stated in #199 and in 1bit-MONSTER/engine#991; nothing in this PR's code changes.

[bong-water-water-bong]

Rebased onto the post-#178 base and re-verified: dee7c9a7d (was edbb2a820).

• Base moved d8bd0ff60 -> 428937ae6 (that is model: Zamba2 on HRX (Mamba-2 layers + 2 shared attention blocks) #178 merged, plus hrx: the residency limit's header says its default is 2048 MiB; the code says 0 (off) #191/hrx: refuse a packed weight layout instead of running it over unpacked bytes #192/context: keep a fused op when only its layer's placement disagrees (-fcd) #194/hrx: cooperative bucket scan, single emit pass and vec4 row reads for hip_top_k_f32 (engine #679) #195/hrx: standalone SET_ROWS dispatch for the QSA mask rebuild (engine #679) #196 and hrx: say once, at load, when a model's nodes fall back to the CPU (engine #991) #197). The rebase conflicted in the five registry files model: Zamba2 on HRX (Mamba-2 layers + 2 shared attention blocks) #178 had just touched; all five were the same shape — both archs adding to one enum/table/switch/struct list — resolved by keeping both, BLACKMAMBA before ZAMBA2 in the mamba-family group. Two non-obvious spots: src/models/models.h's conflict was two structs sharing one trailing body (choosing a side silently deletes a struct; llama_model_zamba2 was reconstructed and diffed byte-for-byte against git show HEAD:), and in gguf-py/gguf/constants.py the two unterminated tensor lists share a trailing ],, so keeping both needs one extra bracket and removal of the leftover (checked with ast.parse, not by eye).
• Diff is unchanged in substance: 10 files, +302. Both archs are present and registered (LLM_ARCH_BLACKMAMBA and LLM_ARCH_ZAMBA2).
• test-llama-archs (CPU, GGML_HRX=OFF, thermal-run): exit 0, blackmamba … MoE … OK (0.00e+00) | OK and zamba2 … Dense … OK (0.00e+00) | OK, so the two ports coexist on the new base.
• Not re-measured: the device (HRX) numbers in the body are from the pre-rebase base. This rebase touched no kernel or dispatch code, but the base gained context: keep a fused op when only its layer's placement disagrees (-fcd) #194/hrx: cooperative bucket scan, single emit pass and vec4 row reads for hip_top_k_f32 (engine #679) #195/hrx: standalone SET_ROWS dispatch for the QSA mask rebuild (engine #679) #196/hrx: say once, at load, when a model's nodes fall back to the CPU (engine #991) #197, which do change placement; I will refresh the pp/tg numbers on the rebased build rather than let the body carry stale evidence.

Ready for a pi-prs re-verdict at dee7c9a7d.

[bong-water-water-bong]

#183 ready for review at dee7c9a

[bong-water-water-bong]

Device re-measurement on the rebased build at dee7c9a7d (HRX0, gfx1151, gpu-hold, llama-bench -dev HRX0 -ngl 99 -p 512 -n 64 -r 3):

check	body was (old base)	now at dee7c9a7d
pp512	1028.07 ± 8.63 t/s	1124.25 ± 1.89 t/s
tg64	44.09 ± 1.31 t/s	49.21 ± 0.60 t/s
GGML_HRX_STRICT=1 placement	145/1179 off (MUL_MAT 73, MUL_MAT_ID 54, GLU 18), 146 splits	identical: 145/1179 off (MUL_MAT 73, MUL_MAT_ID 54, GLU 18), 146 splits
greedy /completion	' Paris, and the capital of France is Paris. Paris is the capital of France'	identical

The load-bearing numbers did not move: same placement, same per-op breakdown, same greedy text — the rebase kept the arch's behaviour exactly. The ~10-12% on pp/tg I am not claiming as this rebase's doing: the earlier numbers were taken on a hot box (die 65-75 C) and this run started at 42 C, so the difference is most likely clock/thermal state, not the base's new commits (#194-#197 touch the Qwen/QSA paths, not these graphs). Treat the placement table and the greedy string as the evidence and the t/s as indicative.

[bong-water-water-bong]

VERDICT: READY at dee7c9a7d

Merge-only rebase: own contribution (merge-base with 1bit/hrx-vulkan-patched .. head, +/- lines only) hashes to 72133bc3a0 at this head and at the head I verdicted (edbb2a820, base d8bd0ff60) — same hash, same 267 lines, same 10 files, +302/−0.

So the review I posted there stands as written: BlackMamba (blackmamba.cpp, llama-model.cpp/llama-model-loader.cpp/llama-model-saver.cpp/llama-arch.* registration, GGUF arch strings blackmamba/mamba2), no AMD/fork file touched, and the SSM scan path is the one gpu-kernels#50 provides for.