MiMo-V2.6-Flash-RL (Xiaomi)

How to use it

Weights are on Hugging Face and require trust_remote_code=True. ~23,000 downloads last month. No licence is stated on the page. The card gives concrete serving recipes rather than just naming vLLM/SGLang.

Recommended sampling: temperature=1.0, top_p=0.95.

SGLang (the card's preferred path; "follow the SGLang MiMo cookbook"):

sglang serve \
  --trust-remote-code \
  --model-path XiaomiMiMo/MiMo-V2.6-Flash-RL \
  --tp 8 \
  --dp 2 \
  --enable-dp-attention \
  --enable-dp-lm-head \
  --mm-enable-dp-encoder \
  --mem-fraction-static 0.65 \
  --chunked-prefill-size 16384 \
  --speculative-algorithm EAGLE \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4 \
  --enable-multi-layer-eagle \
  --reasoning-parser mimo \
  --tool-call-parser mimo \
  --host 0.0.0.0 --port 30000

The recommended flag set implies an 8×TP / 2×DP (16-GPU) topology with DP-attention and DP encoder. Note the custom --reasoning-parser mimo and --tool-call-parser mimo; standard parsers will not handle this model's output.

vLLM lags — the card says to "follow the vLLM MiMo-V2.5 recipe" and warns stable vLLM may not support it, pointing to a pinned image vllm/vllm-openai:mimov25-cu129:

vllm serve XiaomiMiMo/MiMo-V2.6-Flash-RL \
  --tensor-parallel-size 4 \
  --trust-remote-code \
  --gpu-memory-utilization 0.95 \
  --max-model-len auto \
  --reasoning-parser mimo \
  --tool-call-parser mimo \
  --enable-auto-tool-choice \
  --generation-config vllm

Also served hosted via AI Studio, Xiaomi MiMo Open Platform API, and OpenRouter, if you'd rather not host 309B params yourself.

Details the summary omits

• MTP throughput: the 5-layer SWA drafter (DFlash-style) predicts 7 subsequent tokens per forward pass for parallel verification — the concrete speculative-decoding payoff.
• RL scale: fully asynchronous GRPO at 1,568 prompts × 16 rollouts per step, billions of tokens per update.
• Audio path specifics: AudioTokenizer is 24 layers (12 SWA / 12 GA), hidden 1024, 20 RVQ codebooks; the patch encoder does 4 frames per patch, downsampling 25 Hz → 6.25 Hz.
• Backbone layout: first block is global attention + dense FFN; remaining blocks interleave local SWA and global attention, all sparse MoE with no shared experts.

Limits

No benchmark numbers are on the page — it only links leaderboard evaluations (datacurve/deep-swe, terminal-bench 2.1 and 4, hkust-nlp/Toolathlon) without stating scores or baselines, so there is nothing here to judge quality against. No licence is given. The vLLM path is explicitly flagged as possibly lagging behind stable releases, requiring a pinned CUDA-129 image.