From e722af9d59db981670ec09ce15ceab729b17d124 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Tue, 10 Nov 2026 15:12:53 -0400 Subject: [PATCH] docs: 1bit ort-run and ort_run share one parser; a pack's *.onnx may need vendor op domains (refs #2000) #2000 found two documentation gaps while measuring a 1.9 pack. 1. docs/onnx.md now says the two entry points are one function (onebit_ort_run_main; app/main.cpp passes argv through unchanged). Both take --input/--output/--dump-all, which have been there since #1406. The differences are build-side: `1bit ort-run` needs -DONEBIT_ORT=ON, and the standalone ort_run is its own target, which `--target onebit` does not build. "unknown argument --input" from `1bit ort-run` therefore means a 1bit older than #1406. The doc says how to rebuild it, and points at the #1993 target-name trap. 2. docs/onnx.md (model-file traps) and docs/onnx-pack.md say a Ryzen AI pack's *.onnx component can carry custom-domain nodes behind a plain interface. The issue's example is vision.onnx failing on com.ryzenai:DynamicPad, so no plain EP loads it and there is no CPU-EP oracle for such a component. Whether to change the CLI is left to the onnx/ owner. The source shows the flags already match, so this PR documents what the code does. Co-authored-by: bong-water-water-bong Co-Authored-By: Claude Opus 5.5 --- docs/onnx-pack.md | 6 ++++++ docs/onnx.md | 31 ++++++++++++++++++++++++++----- 2 files changed, 32 insertions(+), 5 deletions(-) diff --git a/docs/onnx-pack.md b/docs/onnx-pack.md index 9cb8b645a..02ce44da2 100644 --- a/docs/onnx-pack.md +++ b/docs/onnx-pack.md @@ -28,6 +28,12 @@ execution provider, with weights pre-quantised into that provider's own layout. opaque: the pack decides the quantisation, the kernel, the tiling and the schedule, and the operator that runs on the device is a black box the vendor compiles. +This holds for every `*.onnx` in a pack, not only the transformer. A component whose inputs and outputs +are plain tensors can still carry custom-domain nodes. For example, a 1.9 pack's `vision.onnx` holds +`com.ryzenai:DynamicPad`, and its strategy file lists more rewrites of the same kind. Under the plain +ONNX Runtime EPs it fails at session create, so "the pack ships it as ONNX" does not mean "`ort-run` +can run it" (engine#2000). + We do not execute those operators. Three concrete reasons, in the order they bind: 1. **Licence and footprint.** The VitisAI EP arrives in the AMD wheel, which the contract forbids us diff --git a/docs/onnx.md b/docs/onnx.md index 48955cc7a..faacf8d03 100644 --- a/docs/onnx.md +++ b/docs/onnx.md @@ -484,17 +484,38 @@ build/ort_run --model m.onnx --ep build/ep/libonebit_ep.so \ | `--output NAME=FILE` | repeatable; write that output to that path | | `--dump-all DIR` | write every output as `DIR/.bin` plus `DIR/outputs.txt`, one line per tensor with its dtype, shape, byte count and path | +**Two entry points, one parser.** `1bit ort-run …` and the standalone `build/ort_run …` are the same +function (`onebit_ort_run_main` in `onnx/ort_run.cpp`; `app/main.cpp` hands it the subcommand's +arguments unchanged), so they accept the same flags, `--input`/`--output`/`--dump-all` included +(since #1406). They differ only in how they are built: + +- `1bit ort-run` exists only in a build configured with `-DONEBIT_ORT=ON`. Without that, `1bit` says + `1bit: unknown command 'ort-run'`. +- the standalone binary is its own target. `--target onebit` does not build it, so run + `ninja -C build ort_run`. + +If `1bit ort-run` answers `onebit-ort-run: unknown argument --input`, that `1bit` is older than #1406. +Rebuild it with `--target onebit` (the target is `onebit`, while `1bit` is only its output file name: +engine#1993) and check the binary's mtime against the checkout. + Both kinds of graph keep the same EP ordering story: `--ep` libraries are registered in the order given and the CPU EP is always appended last, so our EP is asked first and whatever it refuses falls through. A graph the EP cannot help is still served — and the EP's `ONEBIT_EP_TRACE=1` log says how many nodes it was offered and how many it claimed, which is the difference between "the plugin was loaded" and "the plugin did work". -Two model-file traps are worth knowing before blaming the runner: a graph exported by python `onnx` -1.23 defaults to **IR version 14** and ONNX Runtime 1.23.2 refuses anything above **11** -(`Unsupported model IR version: 14, max supported IR version: 11`), and a graph with any `MatMul` -whose `K`/`N` no NPU ELF covers simply falls through to the CPU EP — which is the designed behaviour, -not an error. +Three model-file traps are worth knowing before blaming the runner: + +- A graph exported by python `onnx` 1.23 defaults to **IR version 14**, and ONNX Runtime 1.23.2 + refuses anything above **11** (`Unsupported model IR version: 14, max supported IR version: 11`). +- A graph with any `MatMul` whose `K`/`N` no NPU ELF covers simply falls through to the CPU EP. That + is the designed behaviour, not an error. +- **A Ryzen AI pack's `*.onnx` component is not a plain graph, even when its interface is.** For + example, `vision.onnx` in `Qwen3.5_4B_rai_1.9.0_npu` (inputs `pixel_values` f32 and + `image_grid_thw` i64) fails at session create with + `com.ryzenai:DynamicPad(-1) is not a registered function/op`. Its strategy passes rewrite it into + the vendor's custom op domain ([onnx-pack.md](onnx-pack.md)). No plain EP loads it, so such a + component has no CPU-EP oracle here (engine#2000). `onnx/tests/ort_run_generic.sh` (ctest `ort_run_generic`, registered when python `onnx` is present) writes a two-node `MatMul`+`Relu` model, runs this path on it, compares the tensor it writes with