[TRTLLM-17033][feat] Qwen3.8-Flash-Next modeling_v2: CUTEDSL, MTP 1..7, video, FP8/NVFP4 KV, bf16 state, canonical QSA selection, breakable prefill graph, GB300 fast paths

cpp/tensorrt_llm/thop/attentionOp.cpp

@@ -1355,7 +1355,11 @@ void attention(torch::Tensor q, std::optional<torch::Tensor> k, std::optional<to
     op->mRotaryEmbeddingLongMscale = rope_long_m_scale;
     op->mRotaryEmbeddingMaxPositions = rope_max_positions;
     op->mRotaryEmbeddingOriginalMaxPositions = rope_original_max_positions;
+    // Only the FP8 context FMHA kernels read an NVFP4 KV cache, whatever the output type; they do so on SM100-family
+    // GPUs only, so elsewhere an NVFP4 KV cache still fails the FP4 KV check in AttentionOp. MLA keeps its own NVFP4 KV
+    // handling (mUseNvfp4MlaKvCache, mFP8ContextMLA below) and must not pick this up.
     op->mFP8ContextFMHA = is_fp8_out || is_fp4_out || (op->mKVCacheQuantMode.hasFp8KvCache() && use_paged_context_fmha)
+        || (!is_mla_enable && op->mKVCacheQuantMode.hasFp4KvCache() && tensorrt_llm::common::isSM100Family())
         || use_sage_attn;
     // SageAttention block sizes and quantization mode.
     op->mSageAttnNumEltsPerBlkQ = static_cast<int>(sage_attn_num_elts_per_blk_q);

tensorrt_llm/_torch/_experimental/modeling_v2/README.md

@@ -36,10 +36,15 @@ exists to catch.
 
 The decision is one function, `route`, in the family's `routing.py`. It reads
 the facts that fix which code runs -- the checkpoint's config shape, the
-parallel mapping, the GPU's compute capability and the speculative-decoding
-config -- and either returns a target class or raises `NoTarget` with the
-reason in words. Tuning knobs such as `max_batch_size`, `max_num_tokens` and CUDA graph
-settings are not routing inputs: they do not change which code runs, so a
+parallel mapping, the GPU's compute capability, the speculative-decoding
+config, the KV cache algorithm the engine will run with, and the engine choices
+it derives from the `ModelConfig` and the LLM arguments it is also given
+(qwen4_exp: MoE backend, recurrent-state cache dtype, prefill CUDA graph
+backend) -- and either returns a target class or raises `NoTarget` with the
+reason in words. A target declines what its
+constructor would refuse, so that `auto` falls back instead of failing mid-build.
+Tuning knobs such as `max_batch_size`, `max_num_tokens` and the CUDA graph
+bucket lists are not routing inputs: they do not change which code runs, so a
 target claims a configuration regardless of them. `require` therefore promises
 that the target ran, not that a particular knob setting was measured. The model
 loader decides when it is about to build the model, after the built-in model's
@@ -67,7 +72,8 @@ LLM(model=..., modeling_v2=...)  ->  ModelLoader
                                               |
                     _router_index.py          +-- architectures[0] -> routing module
                     models/<x>/routing.py     +-- route(config, mapping, compute_capability,
-                                              |         spec_config, kv_cache_quant_algo)
+                                              |         spec_config, kv_cache_quant_algo,
+                                              |         model_config, llm_args)
                                               |         -> target class, or NoTarget
                                         |
                                      config.modeling_v2_target = class

tensorrt_llm/_torch/_experimental/modeling_v2/_router_index.py

@@ -17,10 +17,12 @@
   architecture family does.
 * The routing module's ``route`` owns the rest of the decision as one
   forward-reading tree over the facts that fix which code runs: the
-  checkpoint's config, the parallel mapping, the GPU's compute capability and
-  the speculative-decoding config. It returns the target class, or raises
-  :class:`NoTarget` with the reason in words. Reading that one file tells you
-  where any configuration lands.
+  checkpoint's config, the parallel mapping, the GPU's compute capability, the
+  speculative-decoding config and the KV-cache algorithm the engine will run
+  with. It also receives the ``ModelConfig`` and the LLM arguments, from which a
+  family derives any other engine choice its targets depend on. It returns the
+  target class, or raises :class:`NoTarget` with the reason in words. Reading
+  that one file tells you where any configuration lands.
 
 Tuning knobs -- ``max_batch_size``, ``max_num_tokens``, CUDA graph settings --
 are deliberately not routing inputs. They do not change which code runs, so a
@@ -195,6 +197,8 @@ def modeling_v2_resolve(config: "ModelConfig", llm_args: "TorchLlmArgs") -> Opti
             _device_compute_capability(),
             config.spec_config,
             _kv_cache_quant_algo(config, llm_args),
+            model_config=config,
+            llm_args=llm_args,
         )
     except NoTarget as why:
         if mode is ModelingV2Mode.REQUIRE:

tensorrt_llm/_torch/_experimental/modeling_v2/catalog/attention/fused_qk_norm_rope.py

@@ -103,6 +103,21 @@ class FusedQkNormRope(OpWrapper):
             "GEMMA_PREFILL_ATOL",
             spec=dict(tokens=8192, gemma_prefill_band=True, **_QWEN_MAIN),
         ),
+        # The main rope run in place on qwen4_exp's fused input projection, whose
+        # row is [q | k | v | gate | index | pad] = 55 heads of 256: everything
+        # after k is passed as v heads, which only sets the row stride.
+        Cell(
+            why="qwen4_exp fused-row main rope, 1 token: 29 v heads only set the row stride",
+            spec=dict(tokens=1, **{**_QWEN_MAIN, "num_heads_v": 29}),
+        ),
+        Cell(
+            why="qwen4_exp fused-row main rope, 128 tokens: decode at max concurrency",
+            spec=dict(tokens=128, **{**_QWEN_MAIN, "num_heads_v": 29}),
+        ),
+        Cell(
+            why="qwen4_exp fused-row main rope, 8192 tokens: prefill, under GEMMA_PREFILL_ATOL",
+            spec=dict(tokens=8192, gemma_prefill_band=True, **{**_QWEN_MAIN, "num_heads_v": 29}),
+        ),
         Cell(
             why="qwen4_exp index rope (mRoPE), 1 token: one k and one v head, the "
             "narrowest layout; 5 warps of an 8-warp block",