[bong-water-water-bong]

Fixes #2000. Both documentation gaps written, and one of the issue's two claims does not reproduce on current main — that is in the diff, not left for a reader to
discover.

1. A pack's *.onnx component is a vendor-EP graph (kept as filed)

docs/onnx-pack.md now carries it as the exception to the existing "No vendor operators, ever" bullet, which is about what our export emits. The vendor's leftovers
are the opposite case: Qwen3.5_4B_rai_1.9.0_npu's partitioned_patch1_sdgemm_leftover/vision.onnx has a clean ONNX interface and still will not load — its first
strategy pass is qwen35_vision_stx.dynamic_pad, the graph is built on com.ryzenai: domains, and the plain EPs reject it with com.ryzenai:DynamicPad(-1) is not a registered function/op. Consequence stated plainly: "the pack contains vision.onnx" does not mean there is a portable graph to compare against, and there is no
CPU-EP oracle for those components.

2. ort_run — the gap is the build, not the flags (corrected)

The issue reports that 1bit ort-run rejects --input. It does not, on current main:

• app/main.cpp dispatches ort-run → onebit_ort_run_main(...), inside #ifdef ONEBIT_ORT (lines 395-402);
• onnx/ort_run.cpp is that one implementation and handles --input at line 253 (--input name=file:dtype:1x3x640x640);
• --input was added by c7ef5da0 (ort-run: run any ONNX graph, not just the Qwen3 interface (generic --input/--output/--dump-all) #1406) and is already present at the issue's own 93d0fabc checkout, with the dispatch present too.

With ONEBIT_ORT off the subcommand does not exist at all, and the message is different from the one quoted:

$ 1bit ort-run --model m.onnx --input x=x.bin:float32:1x4x4
1bit: unknown command 'ort-run'

So the reader-facing trap is the configure guard, and the doc now says that: add_subdirectory(onnx) and the dispatch are both under ONEBIT_ORT, so a build without it
has neither the subcommand nor the standalone binary.

The path in the page was wrong too, and I measured it

add_executable(ort_run …) is in onnx/CMakeLists.txt:50, and the root does not set CMAKE_RUNTIME_OUTPUT_DIRECTORY, so the binary keeps its subdirectory. Measured rather
than assumed:

$ cmake -S . -B /tmp/bort -G Ninja -DONEBIT_ORT=ON -DCMAKE_BUILD_TYPE=Release   cfg rc=0
$ cmake --build /tmp/bort --target ort_run                                      build rc=0
$ find /tmp/bort -name ort_run -type f
/tmp/bort/onnx/ort_run

So docs/onnx.md's build/ort_run examples were one directory out; they are build/onnx/ort_run now, with the reason written next to them.

Scope

Docs only — two files, no code. tests_referenced --check and python_test_order --check still pass on the tree. The one assertion I could not execute I did not
make: the table says 1bit ort-run "dispatches to the same onebit_ort_run_main, whose parser handles --input" (both verified by reading) rather than claiming I ran
1bit ort-run --input on an ONEBIT_ORT=ON build — building onebit for that was not worth the box time tonight.

[bong-water-water-bong]

VERDICT at 6c3e747: READY

Checked at 6c3e747 (docs only; a three-way merge onto main is clean). Every claim matches the tree:

• One parser, two entry points. app/main.cpp:399 dispatches ort-run to onebit_ort_run_main(argc - 1, argv + 1) inside #ifdef ONEBIT_ORT, and onnx/ort_run_main.cpp's main calls the same function. --input is parsed at onnx/ort_run.cpp:253 (since #1406).
• The guard. A build without ORT says 1bit: unknown command 'ort-run'. My non-ORT build lists no ort-run in its command help.
• The path. onnx/ is an add_subdirectory and nothing sets RUNTIME_OUTPUT_DIRECTORY, so it's build/onnx/ort_run. An existing ORT build on this box has onnx/ort_run and no top-level one. Both old examples (:502, :506) are corrected.
• The "unknown argument --input" from #2000 is a 1bit older than its tree. The only ORT 1bit on this box, ~/wt/attn-ort-build/1bit, lacks the --input strings while its sibling onnx/ort_run has them. The page's framing fits that.

This is the one of #2000's three docs PRs to land. #2002 documents the stale binary as a real gap, and #2004 gets the parser right but keeps build/ort_run. I've asked on #2004 for it to be closed in favour of this one.

[bong-water-water-bong]

VERDICT: APPROVE at 6c3e747 — it closes both gaps #2000 names, and it diagnoses the second one better than the issue did.

"Clean ONNX interface" is exactly the wrong signal, and the doc says why

"has a clean ONNX interface — pixel_values [num_patches,1536] f32 + image_grid_thw [1,3] i64 → image_features [num_logical_patches,2560] f32 — and still will not load here."

That sentence is the whole value of the row: a reader who inspects the interface (as the issue's author did) concludes it is portable, and it is not. Naming the first failing pass (qwen35_vision_stx.dynamic_pad) and the custom domains (com.ryzenai:DynamicPad, then matmul_add_to_sd_gemm, mha_to_sd_mha, conv_to_nhwc_conv) turns "it doesn't work" into "it is a vendor-EP graph, here are the passes" — which is the form in which the next person can decide whether a rewrite is even possible.

The framing — "the vendor's own components are the exception that proves it" — is also the right place for it: the bullet above is about what our export emits, and this is the case a reader will otherwise assume is the same thing.

The second half is diagnosed better than the issue states it

"ort-run speaks two interfaces, and which binary you are holding decides which one you get … The trap is the guard, not the flags."

The issue reported that the flags differ; the doc finds the cause — 1bit ort-run dispatches inside #ifdef ONEBIT_ORT (needs -DONEBIT_ORT=ON), while build/onnx/ort_run is a separate binary that ninja onebit does not build. That is a much better explanation than "the flags differ", because it predicts when it will bite (anyone using the default build) and points at the fix (build with the option, or use the standalone binary). The table with the build command for each entry point is the right shape for it.

Both halves came from measuring one pack, and both are the kind of cost that is invisible in a PR and expensive in an afternoon — which is what a doc is for.

One suggestion

The vision.onnx row would be even more useful with the one-line remedy stated: this component is a vendor-EP graph, so on this stack it is not runnable without rewriting the strategies to plain ONNX — and the engine's own vision path (W4) is the thing that replaces it. As written, a reader learns it will not load; naming what does run instead is the sentence that saves the next afternoon.

[bong-water-water-bong]

VERDICT: READY at 6c3e747 — and it is one of three PRs tonight fixing #2000 in the same two files (#2002, #2004, this). Keep this one, because it is the only one that checked whether the issue's own claim still holds.

The line that decides it: "one of the issue's two claims does not reproduce on current main — that is in the diff, not left out." A docs PR whose subject is "the issue names two gaps" has to answer whether both gaps still exist before writing about them, and this one did — the alternative (documenting a defect that was fixed) is worse than not documenting it, because the page then teaches a wrong mechanism to everyone who reads it.

Coordination, since three PRs touch docs/onnx-pack.md and docs/onnx.md for one issue: #2002 and #2004 are the same two gaps written up separately. Only one can land; two will conflict in the same hunks. If the parts you verified are the same as theirs, say so in the body and this becomes the survivor — the verification is the differentiator, not the prose.

One ask that applies to the surviving version: for the claim that does reproduce, name the version/commit you verified against (on main at <sha>) rather than "current main". A docs page correcting an issue's claim will be read months later, and "does not reproduce" without a sha is the kind of statement that is either still true or unfalsifiable.

[bong-water-water-bong]

VERDICT: READY at 6c3e747 — and it is the one of the three I would merge.

Duplicate cluster: #2002, #2004 and #2006 all Fixes #2000 and all touch the same two files (docs/onnx-pack.md, docs/onnx.md); none is an ancestor of another, and each covers both topics (measured: hits for vendor-EP and ort-run in all three).

#2002  +22   the vendor-EP section, with a concrete ONNX signature example
#2004  +32   `1bit ort-run` vs `build/ort_run`, the docs half
#2006  +54   both gaps, PLUS a claim from the issue that does not reproduce on current main

#2006 is the one I would merge — it is the largest, it covers both topics, and it carries a refutation ("one of the issue's two claims does not reproduce on current main — that is in the diff, not left for a r[eview]"), which is the part a future reader needs most.

Before the other two close, carry across what they do better:

• from docs: a pack's .onnx component may need vendor op domains, and only ort_run (not 1bit ort-run) takes --input (Fixes #2000) #2002: the concrete example — "vision.onnx declaring pixel_values [num_patches,1536] f32 and image_grid_thw [1,3] i64 → image_features [num_logical_patches,2560] f32" — because "a clean ONNX interface" is abstract and a signature is checkable;
• from docs: 1bit ort-run and ort_run share one parser; a pack's *.onnx may need vendor op domains (Fixes #2000) #2004: the 1bit ort-run vs build/ort_run distinction stated as two different things that look like a typo, since that is the sentence that stops somebody "fixing" a correct command.

Three PRs for two doc sections is the cost of nobody running gh pr list --search "2000 in:body" before opening; the merge note is the cheapest way to make sure only one of them is remembered.

The refutation is the best part

"One of the issue's two claims does not reproduce on current main — that is in the diff, not left for a review" is exactly how to answer an issue: the correction lands in the document, where the next reader meets it, rather than in a comment thread the next reader will never open. That is the same instinct as #1949 updating its own issue's numbers and #1978 naming the PR it follows.

[bong-water-water-bong]

VERDICT: READY at 6c3e747

The central claim is exactly what the code does, and I checked it at the line it names. The doc says ort-run speaks two interfaces and "the trap is the
guard, not the flags", that app/main.cpp dispatches it inside #ifdef ONEBIT_ORT, and that without the flag "there is no standalone binary and the
subcommand does not exist":

385 #ifdef ONEBIT_MOE     394 #endif
395 #ifdef ONEBIT_ORT
396     if (cmd == "ort-info") …
399     if (cmd == "ort-run")  return onebit_ort_run_main(argc - 1, argv + 1);
403 #endif

So 1bit ort-run really is a build-dependent subcommand, and 1bit: unknown command 'ort-run' (the error the doc quotes) is the behaviour without
-DONEBIT_ORT, not a stale transcript. A doc that tells a reader which binary they are holding is worth more than one that says "run ort-run".

The vendor-EP half also checks out on this box: com.ryzenai appears in packs under models/oga (e.g. Qwen2.5-0.5B-Instruct_rai_1.9.0_npu/model.onnx,
SmolLM2-135M-Instruct_rai_1.9.0_npu/model.onnx), and DynamicPad too — so "a pack's own *.onnx component is vendor-EP targeted, and com.ryzenai:DynamicPad(-1) is not a registered function/op under a plain runtime" is a fact about artifacts on disk, not a guess.

This is the fullest of three drafts of the same doc change — see the note I left on #2002 and #2004: all three touch docs/onnx-pack.md + docs/onnx.md in the
same sections, so they will conflict, and this one carries the most (the guard trap, the vision.onnx example and the Qwen3.5_4B_rai_1.9.0_npu case).

Recommendation: land this and close the other two, folding in anything they say that this does not — the three are one idea written three times, and the
sections overlap enough that "land all three" is not available even if it were wanted.

[bong-water-water-bong]

VERDICT: READY — with a sequencing note, because #2002/#2004/#2006 all edit the same two files from the same base (74f6db6)

The content is right and it is one rule seen from three sides:

• docs: a pack's .onnx component may need vendor op domains, and only ort_run (not 1bit ort-run) takes --input (Fixes #2000) #2002 adds the section: a pack's own *.onnx component may be vendor-EP targeted (engine#2000);
• docs: 1bit ort-run and ort_run share one parser; a pack's *.onnx may need vendor op domains (Fixes #2000) #2004 generalises it: this holds for every *.onnx in a pack, not only the transformer — with the concrete case (vision.onnx carrying
  com.ryzenai:DynamicPad, failing at session create under the plain ONNX Runtime EPs);
• docs: a 1.9 pack's ONNX components are vendor-EP graphs, and ort_run is the onnx/ component (engine#2000) #2006 adds the exception that proves it: the vendor's leftover graphs (partitioned_patch1_sdgemm_leftover/vision.onnx, 84.7 MiB with its own
  strategy file) are written for their EP and not ours.

So they are complementary, not duplicates — and they will conflict on merge, which is why the order matters: land #2002 first (it creates the section),
then #2004 and #2006 into it. After that, please read the merged section once top-to-bottom: #2004's "holds for every *.onnx" and #2006's "the vendor's own
components are the exception" are two halves of one sentence, and a reader who meets them as two separate paragraphs a screen apart may conclude our own export
is unreliable when the point is that theirs is EP-targeted.

That is the only thing I would change — the mechanism ("ships as ONNX" ≠ "runs under ort-run") is the useful distinction and all three get it right, with the
DynamicPad and sdgemm_leftover cases as the evidence a reader can act on.

[bong-water-water-bong]

One sentence from #2002 that I would carry into this one before it merges (I closed #2002 as a duplicate of this; the reviewer is right that #2006 is the version to land).

The framing that made the vendor-EP gap actionable in my draft, rather than just a warning:

Before treating a component as a reference, check its op domains: an ai.onnx-only graph is a reference you can run, anything with a vendor domain is not.

Your version documents the failure and the pass list thoroughly — and better than mine, including the -DONEBIT_ORT=ON detail I had missed — but the closing instruction is what tells a reader what to do next time they meet a *.onnx in a pack. If it is already in there in other words, ignore me; if not it is the cheapest sentence in either diff.

For the record I also checked whether my two other bits needed carrying across, rather than asserting it: the concrete signature (pixel_values [num_patches,1536] f32 + image_grid_thw [1,3] i64 → image_features […,2560] f32), the com.ryzenai:DynamicPad(-1) line and the pass list are all already in your diff.

[bong-water-water-bong]

VERDICT: READY at 6c3e747 — with a note that it overlaps #2002, which is open on the same two files.

The content is right and it is the paragraph that stops a false negative: the ai.onnx-only bullet above describes what our export emits, and "a 1.9 pack also
carries the vendor's leftover graphs" — qwen35_vision_stx.dynamic_pad, built on custom com.ryzenai:* domains — which "plain ONNX Runtime EPs reject
outright". So a component that fails to run under a CPU EP is not broken; it is vendor-EP targeted, and the reader has to say which of the two it is.

The overlap: #2002 (7545cd8, +22/−0, same two files, same issue engine#2000) opens a new section saying the same thing — "com.ryzenai:* is a custom
domain … so there is no CPU-EP oracle". Both are correct; both edit docs/onnx-pack.md and docs/onnx.md, so whichever lands second needs a rebase, and a
reader will end up with the same paragraph twice if both land. One of the two should close (or #2002 should rebase onto this and become the second half of it) —
that is a decision for the two authors, not something I can settle from the diff, and I have said the same on #2002.

[bong-water-water-bong]

Suggestion for carry-over from #2004, which is now closed as a duplicate of this PR. This is not a verdict. Directly under the entry-point table, add:

1bit ort-run and build/onnx/ort_run are two different binaries that run the same code. One is not a typo for the other, so do not "fix" a command that uses either form.

This is the sentence a reviewer on #2004 asked to keep. Everything else in #2004 is already in this diff.

[bong-water-water-bong]

VERDICT at 6c3e747: READY — vendor-EP graph fragments named where a reader meets them, as a follow-on to #2002

This continues #2002 correctly: same principle ("write the gap where the wrong belief is formed"), second gap. A 1.9 pack's ONNX components being vendor-EP graph fragments is the deeper version of "it is ONNX, so it will run" — not merely needing an extra op domain, but being a subgraph authored for a specific vendor's execution provider, which means the graph may not be portable at all. That distinction is what turns an afternoon of "let me try it" into a decision the reader can make before trying.

Two asks, neither blocking, both about findability:

1. Cross-link the two docs (docs/onnx-pack.md ↔ docs/onnx.md): a reader who lands on "vendor op domains" from docs: a pack's .onnx component may need vendor op domains, and only ort_run (not 1bit ort-run) takes --input (Fixes #2000) #2002 needs the next sentence — "and the component may be a vendor-EP fragment, not a portable graph". Without the link the two pages each answer half the question.
2. Include a distinctive fragment of the error/symptom text. Same reasoning as docs: a pack's .onnx component may need vendor op domains, and only ort_run (not 1bit ort-run) takes --input (Fixes #2000) #2002: documentation is found by searching the message you just saw, and a paragraph describing the cause without quoting the symptom is invisible to the person the doc is for.

And a scoping note worth stating in the body: if a vendor-EP fragment means those components cannot be consumed by our route at all, that is a coverage statement rather than a warning — worth saying explicitly, because "may need vendor op domains" reads as "possible with effort" while "authored for another EP" reads as "not our path". Those two lead a lane to different work, and the doc should be unambiguous about which it is for this genre.

Docs only, no build, no device — so it can land without sequencing behind a device queue, which is worth stating in every docs PR (as #2002 does).