DeepSeek-v4.1-Flash

[wtdcode]

196G CPU offload engram and ~320 G weights.

This needs at least A6000 x 8.

[codender]

Come on! I have A100 x 8.

[chasezcz]

BRO!!! WE NEED DS-V4.1-FLASH !!!

[qigi-ai]

Please consider adding DeepSeek-V4.1-FLASH support to the A100 branch — many of us still rely on it, and it would genuinely mean a lot. Thanks for all your work!

[xlg-go]

BRO!!!
A800 * 8

[RoneChen]

BRO!!! V4.1!!! THX!!!

[wtdcode]

Hi, thanks for your passion. I have kick-started the port process.

However, do note that since even upstream vllm does not have day0 support, please expect the port might take slightly longer.

Btw, if you like this project, consider giving me a star and letting your collegues know =p.

[wtdcode]

Update: Get the first request working already.

[wtdcode]

v0.13.0 is in the process of being built. If everything works fine, it should be available with v4.1 support in next 4-5 hours.

[hauck-jvsh]

Is this build already offloading the ngram?

[hwzz3311]

Hey bro, does the image you compiled this time support quantized models? If so, any recommendations?

[wtdcode]

Is this build already offloading the ngram?

Yes.

[wtdcode]

Hey bro, does the image you compiled this time support quantized models? If so, any recommendations?

Most of the DeepSeek-V4.1 weights are already in FP4.

[veritynoob]

which model should i use? in a100*8 origin deepseek-ai/DeepSeek-V4.1-Flash? or LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4?

[wtdcode]

I updated recipe at https://github.com/wtdcode/vllm-backport/blob/master/README.md#deepseek-v41-flash

Please use the official weights.

[halexan]

Any speed performence tests?