Ling-3.0-flash-VL (InclusionAI/Ant Group)

How to use it

The summary says serving recipes exist; here are the specifics an engineer needs before committing. Thinking mode is on by default via the chat template — disable it per request with "chat_template_kwargs": {"enable_thinking": false}. Recommended sampling for serving is temperature=1.0, top_p=0.95, top_k=20 (note: differs from the temperature=0.6 listed as the model's general default). Reasoning and tool-call parsers resolve to ling3.

Hardware tiers are explicit. The SGLang recipe with full 256K context (via YaRN) targets 4× 141GB-class GPUs (H20-3e / H200) or 4-GPU Blackwell (B300/GB300); on 80GB cards (H100/H800) you must scale to --tp 8.

SGLang:

docker pull lmsysorg/sglang:dev-Ling-3.0-flash-VL

docker run --rm --gpus all --ipc=host --shm-size 32g \
  -p 30000:30000 \
  -e HF_TOKEN=<your-hf-token> \
  lmsysorg/sglang:dev-Ling-3.0-flash-VL \
  env SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 \
  python3 -m sglang.launch_server \
    --model-path inclusionAI/Ling-3.0-flash-VL \
    --tp 4 \
    --context-length 262144 \
    --json-model-override-args '{"rope_scaling":{"rope_type":"yarn","factor":2.0,"rope_theta":6000000,"partial_rotary_factor":0.5,"original_max_position_embeddings":131072}}' \
    --trust-remote-code \
    --reasoning-parser auto \
    --tool-call-parser auto \
    --host 0.0.0.0 --port 30000

vLLM does not run on stock vLLM — it requires a custom fork, plus MoE/Mamba-specific flags:

git clone https://github.com/inclusionAI/vllm-ling-v3.git
cd vllm-ling-v3
VLLM_USE_PRECOMPILED=1 uv pip install --editable . --torch-backend=auto

vllm serve "$MODEL_PATH" --port "$PORT" --trust-remote-code \
    --served-model-name auto --tensor-parallel-size 4 \
    --gpu-memory-utilization 0.85 --enable-prefix-caching \
    --mamba-cache-mode align --enable-auto-tool-choice \
    --tool-call-parser ling3 --reasoning-parser ling3

Video uses {"type": "video_url", "video_url": {"url": "..."}} in the same message shape as images. ~11.5K downloads last month.

Evidence

The Terminal-Bench 2.1 setup behind the agentic claims: Artificial Analysis protocol, default Terminus 2 harness, 2-hour timeout, JSON parser in preserve-thinking mode, 3 runs per task (mean), decoding at temperature=1.0, max_new_tokens=32K, 256K context. (The page states the protocol but does not print the resulting score here.)

Limits

No licence is stated on this page — a separate training-content disclosure is referenced but not the usage terms/licence, so verify before adoption. The --mamba-cache-mode flag and custom fork requirement mean the hybrid KDA/MLA backbone is not yet upstream in vLLM; expect maintenance friction tracking that fork.