From 7545cd862d5faafa3e4deb96f3e57585c3834b22 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Tue, 10 Nov 2026 15:11:01 -0400 Subject: [PATCH] docs: a pack's .onnx component may need vendor op domains, and only ort_run (not `1bit ort-run`) takes --input (Fixes #2000) Two gaps that cost real time while measuring a 1.9 pack, neither a code bug: 1. "It is ONNX" was read as "it will run". A pack's side component can declare a clean ONNX interface (`vision.onnx`: `pixel_values [num_patches,1536] f32`, `image_grid_thw [1,3] i64` -> `image_features [num_logical_patches,2560] f32`) and still fail to load on the plain execution providers with `com.ryzenai:DynamicPad(-1) is not a registered function/op`. `com.ryzenai:*` is a custom domain, so there is no CPU-EP oracle: check the op domains before treating a component as a reference. `docs/onnx-pack.md`. 2. The arbitrary-graph mode (`--input/--output/--dump-all`) is the **standalone `ort_run`** binary, which `ninja onebit` does not build; `1bit ort-run` rejects those flags with `unknown argument --input`. Said at the top of the section that documents the flags, rather than leaving the reader to discover which of the two entry points it applies to. `docs/onnx.md`. Docs only; no code, no build, no device. --- docs/onnx-pack.md | 16 ++++++++++++++++ docs/onnx.md | 6 ++++++ 2 files changed, 22 insertions(+) diff --git a/docs/onnx-pack.md b/docs/onnx-pack.md index 9cb8b645a..848f81013 100644 --- a/docs/onnx-pack.md +++ b/docs/onnx-pack.md @@ -99,6 +99,22 @@ model. | gate/up `1024×3072` | CPU EP | same | | lm_head `1024×151936` | CPU EP | needs a vocab-blocked kernel (N far past one ELF); leave on the CPU until the FFN set is proven | +### A pack's own `*.onnx` component may be vendor-EP targeted (engine#2000) + +"It is ONNX" does not mean "it will run". A 1.9 pack's side components look portable — a clean ONNX +interface, e.g. `vision.onnx` declaring `pixel_values [num_patches,1536] f32` and `image_grid_thw [1,3] i64` +→ `image_features [num_logical_patches,2560] f32` — and still will not load under the plain ONNX Runtime +execution providers: + +``` +Fatal error: com.ryzenai:DynamicPad(-1) is not a registered function/op +``` + +`com.ryzenai:*` is a custom domain (`DynamicPad`, `matmul_add_to_sd_gemm`, `mha_to_sd_mha`, +`conv_to_nhwc_conv` in that pack's `*_strategy.yaml`), so there is **no CPU-EP oracle**: the component has +to go through its compiled `partitioned/` form. Before treating a component as a reference, check its +op domains — an `ai.onnx`-only graph is a reference you can run, anything with a vendor domain is not. + ## What the pack does not do, deliberately - **KV cache: opt-in, not the default.** The default export re-runs the whole sequence per step, so its diff --git a/docs/onnx.md b/docs/onnx.md index 48955cc7a..923abc30e 100644 --- a/docs/onnx.md +++ b/docs/onnx.md @@ -463,6 +463,12 @@ silent fall back cannot be mistaken for a device pass. ## Any graph: `ort-run --input` +> **Two entry points, and only one of them has `--input`** (engine#2000). The arbitrary-graph mode below is +> the standalone `ort_run` binary — build it with `ninja ort_run` (`ninja onebit` does **not** build it), then +> run `build/ort_run …`. The habitual `1bit ort-run` subcommand does not accept `--input`, `--output` or +> `--dump-all` and answers `onebit-ort-run: unknown argument --input`. If a build that looks complete has no +> `build/ort_run`, that is why. + `ort-run` speaks two interfaces. With `--ids` it is the Qwen3 language model of [Qwen3-0.6B](#qwen3-06b) below. With `--input` it is **any** ONNX graph: nothing about the graph's names, shapes or roles is assumed, so a vision model, an audio model and a one-op toy all take the