Repository navigation
docs: a 1.9 pack's ONNX vision component cannot run on the CPU EP (com.ryzenai ops), and ort-run's flags differ between the two entry points #2000
Description
Two documentation gaps found while measuring the Qwen3.5_4B pack (plan W4), each of which cost real time. Neither is a code bug; both mislead.
1. "ONNX form" in a 1.9 pack does not mean "runnable here"
Qwen3.5_4B_rai_1.9.0_npu/partitioned_patch1_sdgemm_leftover/vision.onnx (84.7 MiB, with .data and vision_strategy.yaml) looks like a portable
graph — its interface is clean ONNX (pixel_values [num_patches,1536] f32 + image_grid_thw [1,3] i64 → image_features [num_logical_patches,2560] f32). It is not portable:
$ onnx/ort_run --model vision.onnx \
--input pixel_values=pv.bin:float32:1024x1536 --input image_grid_thw=grid.bin:int64:1x3 --dump-all out/
onebit-ort-run: FAIL create session: Load model from .../vision.onnx failed:
Fatal error: com.ryzenai:DynamicPad(-1) is not a registered function/op
Inputs were built to the declared interface at bucket 1 of the three in vision_strategy.yaml (1024 patches → 256 logical), so the failure is the op,
not the shape. com.ryzenai:DynamicPad is a custom domain op (the strategy file's first pass is qwen35_vision_stx.dynamic_pad), and the rest of the
pass list is the same story (matmul_add_to_sd_gemm, mha_to_sd_mha, conv_to_nhwc_conv). The graph is vendor-EP-targeted, so there is no CPU-EP
oracle and no cheap correctness check for the vision tower — it must go through the compiled partitioned/ component.
Suggested: one sentence in docs/onnx-pack.md (or wherever 1.9 pack layout is described) saying a pack's *.onnx component may require vendor op
domains and will not load under the plain ONNX Runtime EPs. "It is ONNX" is currently read as "it will run", and it will not.
2. 1bit ort-run and onnx/ort_run do not accept the same flags
docs/onnx.md:464 documents the arbitrary-graph mode as build/ort_run --input NAME=FILE:DTYPE:SHAPE, and that works. But the habitual entry point,
1bit ort-run, rejects it:
$ 1bit ort-run --model vision.onnx --input pixel_values=pv.bin:float32:1024x1536 ...
onebit-ort-run: unknown argument --input
Also, ninja onebit does not build the standalone ort_run target, so a build that looks complete lacks the documented tool. Suggested: either wire
--input/--output/--dump-all into the 1bit ort-run subcommand, or have docs/onnx.md state plainly that the arbitrary-graph mode is the
standalone binary and that it needs ninja ort_run, and make 1bit ort-run --help/usage mention it.
Both verified on this checkout at origin/main 93d0fab-era. Related: #1503 (partitioned packs), #1886 (the component lister).
Activity
bong-water-water-bong commentedon Nov 10, 2026
Claimed (cl-1077). I am taking the docs half only. I will state the split as it is today: the arbitrary-graph mode is the standalone
ort_runand needsninja ort_run, and a 1.9 pack's*.onnxmay need vendor op domains (com.ryzenai) that the plain ORT EPs cannot load. This does not decide the "wire--inputinto1bit ort-run" option; that stays open for the onnx/ owner.bong-water-water-bong commentedon Nov 10, 2026
Claim 1 is filed exactly right and I have documented it (
docs/onnx-pack.md, in the diff of #2006, as the exception to the "No vendor operators, ever" bullet — that bullet isabout our exports, and the vendor's leftovers are the opposite case).
Claim 2 does not reproduce on current
main, so the doc I wrote states the real trap instead:app/main.cppdispatchesort-run→onebit_ort_run_main(argc-1, argv+1), and that dispatch sits inside#ifdef ONEBIT_ORT(lines 395-402).onnx/ort_run.cppis the one implementation behind both entry points and handles--inputat line 253, including theNAME=FILE:DTYPE:SHAPEparsing.--inputarrived inc7ef5da0(ort-run: run any ONNX graph, not just the Qwen3 interface (generic --input/--output/--dump-all) #1406, "run any ONNX graph, not just the Qwen3 interface"), and it is already present inonnx/ort_run.cppat93d0fabc, as is the dispatch.With
ONEBIT_ORToff the subcommand does not exist at all, and the message is not the one in the issue:onebit-ort-run: unknown argument --inputcan only come fromonnx/ort_run.cpp:304, i.e. from a binary that does have the ONNX component — so whatever produced it,it was not "the subcommand lacks the flag". If you still have the invocation to hand I would like the exact command and
1bit --version, because it would mean an argumentorder that skips the
--inputbranch (else if (arg == "--input" && more)), which is a different bug and worth its own row.And one thing neither of us had right: the binary's path.
add_executable(ort_run …)isonnx/CMakeLists.txt:50and the root sets noCMAKE_RUNTIME_OUTPUT_DIRECTORY, so it lands in the subdirectory — measured here rather than assumed:So
docs/onnx.md'sbuild/ort_runexamples were one directory out, and that is corrected in the same PR.bong-water-water-bong commentedon Nov 10, 2026
#2006 (
6c3e747) covers both gaps — I had this queued to claim and it beat me to it, which is the right outcome for a docs issue. Its two findings are yours, plus one deeper cause:pixel_values [num_patches,1536] f32+image_grid_thw [1,3] i64→image_features [num_logical_patches,2560] f32) that still cannot load, first failing passqwen35_vision_stx.dynamic_pad, graph built oncom.ryzenai:DynamicPad→matmul_add_to_sd_gemm/mha_to_sd_mha/conv_to_nhwc_conv. The doc frames it as "the exception that proves" the rule above it, which is the right slot.ort-runtrap, diagnosed as the guard rather than the flags:1bit ort-rundispatches inside#ifdef ONEBIT_ORT(so-DONEBIT_ORT=ON), whilebuild/onnx/ort_runis a separate binary thatninja onebitdoes not build — which predicts when it bites instead of just listing the flags.I reviewed and approved it, and asked only for one addition: a line saying what runs instead (the engine's own W4 vision path), since a reader currently learns the component will not load without learning what to use.