x.comthreadGunjan N Ganatra (@gnganatra) on X
LightSeek Foundation announces 685 output tokens/s per user on Alibaba's Qwen3.8-Flash-Next. They ran their TokenSpeed serving engine on 4× NVIDIA GB300 with NVFP4 quantisation, and link a blog on the kernel and serving optimizations plus the GitHub repo. In the same thread, a user asks the team to bring similar optimizations to NVIDIA DGX Spark users running Qwen3.8-27B on vLLM, and to the upcoming Qwen4-27B. The thread points to a new open inference engine and shows demand for consumer-class DGX Spark serving.
TL;DR LightSeek's TokenSpeed engine reaches 685 tok/s/user on Qwen3.8-Flash-Next (4× GB300, NVFP4); a user asks for DGX Spark and Qwen3.8-27B vLLM support.
Organisations named here
- LightSeek Foundation
- Alibaba Qwen
- NVIDIA
- Qwen
- vLLM
Technologies named here
- TokenSpeed
- Qwen3.8-Flash-Next
- Qwen3.8-27B
- vLLM
- NVFP4
- Qwen4-27B
- NVIDIA DGX Spark
- NVIDIA GB300
Related in inference
arxiv.orgpaperMinimally Invasive Steering of Language Models
x.comthreadSamuel Zeng (@SamuelZengML) on X
huggingface.coarticle