[megatron] 5/n towards Kimi K2.6: Kimi K2.5-family (KimiK25ForConditionalGeneration) support with INT4 QAT + LoRA

[casper-hansen]

What

• KimiK25TextBridge: DeepSeek-V3 bridge adapted to the unified VL checkpoint (text_config + language_model. weight prefix); text-only training with the vision tower left frozen in the inference engine (language_model_only), with a clear error for full-VLM requests.
• vLLM: declare KimiK25ForConditionalGeneration LoRA-capable (patch applied in the SkyRL worker extension import; the DeepSeek decoder already supports LoRA). The worker-side LoRA gate is an instance-level runtime Protocol check that requires every SupportsLoRA member (is_3d_moe_weight, is_non_gated_moe, lora_skip_prefixes too); we mirror the DeepSeek decoder's declarations and pin it with a protocol-membership test.
• Forward HF cache/auth env vars (HF_HOME & co) to Ray workers so model paths resolve identically on worker actors.
• Truncated-real-model GPU integration test: bridge dispatch, provider construction, weight round-trip, LoRA export (incl. capacity-normalized expert LoRA).

Why

Kimi K2.5-family checkpoints (K2.6, K2.7-Code) are unified VL checkpoints whose language model is DeepSeek-V3 architecture; there is no upstream bridge for the arch name, so AutoBridge dispatch fails without this.

Stacked on 4/n (#2026) -- the GPU test exercises normalize_moe_lora; the diff includes 4/n until it merges. Marked draft until then. Part of the Kimi K2.x series (follow-up to #1862).

Made with Cursor

[casper-hansen]

Updated for the vLLM 0.26 bump (#1854, now on main) and merged main in (via the 4/n base branch):

• vLLM 0.26 added an annotation-only lora_manager member to the SupportsLoRA protocol. Native models are nominal subclasses and skip the structural check, but the patched KimiK25ForConditionalGeneration is checked structurally via a runtime-checkable Protocol isinstance(), which requires every member to exist on the class. The patch now sets cls.lora_manager = None (the documented "not yet set" state); the worker's LoRA mixin assigns the real manager at runtime.
• KimiK25TextBridge.build_conversion_tasks now forwards *args, **kwargs so it stays compatible with megatron-bridge 0.7.0 ([deps][megatron] bump megatron-bridge to 0.7.0 and megatron-core to 0.20.0 #2042), which grows the base signature with a weight_dtype keyword. If [deps][megatron] bump megatron-bridge to 0.7.0 and megatron-core to 0.20.0 #2042 lands first, its module-level None-task filter makes our per-bridge filter redundant but harmless; if this lands first, the filter is still needed on the current bridge pin. Heads-up that [deps][megatron] bump megatron-bridge to 0.7.0 and megatron-core to 0.20.0 #2042 and this PR both touch the same import block in model_bridges.py, so whichever merges second will need a trivial conflict resolution.

The rest of the K2.5 bridge (mapping re-rooting via get_common_mapping_list, _TextConfigShim, worker-wrap patch application) is unchanged: the 0.7.0 get_common_mapping_list keeps its signature, and hf_param is still only read at resolve/export time, so the post-construction prefixing stays valid.