Qwen3.8-27B (Apache-2.0, multimodal dense, 262k ctx)

What it claims

Qwen3.8-27B is a 27B dense vision-language model (images and video) built on the Qwen3.5 architecture. The card positions it as the compact, deployable member of the Qwen3.8 generation, aimed at coding and long-horizon agent work. Native context is 262,144 tokens, extensible to 1,000,000 via YaRN. A hosted version on Qwen Cloud is announced with 1M context by default and built-in tools, but marked "coming soon."

How it works

• Size and shape. 64 layers, hidden dimension 5120, FFN intermediate 17,408, padded vocabulary 248,320.
• Layer pattern. Every 4th layer is full attention; the rest are linear-attention.
• Gated DeltaNet layers: 48 V heads and 16 QK heads, head dimension 128.
• Gated Attention layers: 24 Q heads and 4 KV heads (GQA), head dimension 256, RoPE on 64 dimensions (partial_rotary_factor 0.25).
• MTP. Trained with multiple multi-token-prediction steps.
• Preserved thinking. By default, thinking blocks from all past turns are kept in context. The card says this improves KV-cache reuse in agent loops.

Evidence

The fetched text carries only the evaluation protocols, not the scores. Most coding and agent benchmarks were run in the Claude Code harness at temp=1.0, top_p=0.95, with a 256K context window:

• SWE-bench Pro: problematic tasks were corrected and all baselines re-run. The exception is Opus 4.6 Max, which uses its officially reported score.
• NL2Repo-Bench: pip install, pip download and git clone of the target repository were blocked to prevent reward hacking.
• QwenSWEBench: avg@3, 8-hour timeout, max_tokens=32,768.
• HLE: judged by GPT-4o.
• MathVision and CharXiv: scored on ground-truth annotations the authors corrected themselves.
• Prompting asymmetry on MathVision: Qwen used a single fixed prompt, while competitors got the better of two prompt variants.

QwenSWEBench, CoWorkBench and RecreationBench are in-house benchmarks.

How to use it

Sampling.

• Thinking: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0
• Non-thinking: temperature=0.7, top_p=0.80, top_k=20, presence_penalty=1.5

Request controls.

• reasoning_effort accepts xhigh (the default), medium or low.
• Disable thinking or preserved thinking with chat_template_kwargs: {"enable_thinking": False} or {"preserve_thinking": False}.
• On Qwen Cloud, pass these as top-level fields instead of inside chat_template_kwargs.

Output budget. For agentic tasks the card suggests up to 262,144 reasoning tokens and 131,072 final-answer tokens.

1M context with vLLM:

VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 vllm serve ... --hf-overrides '{"text_config": {"rope_parameters": {"mrope_interleaved": true, "mrope_section": [11, 11, 10], "rope_type": "yarn", "rope_theta": 10000000, "partial_rotary_factor": 0.25, "factor": 4.0, "original_max_position_embeddings": 262144}}}' --max-model-len 1000000

SGLang and TokenSpeed have equivalent commands, using SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 ... --json-model-override-args ... --context-length 1000000 and TOKENSPEED_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 tokenspeed serve ... --hf-overrides .... Scale factor to your real length; for example, use 2.0 for about 524K.

Hour-scale video. The shipped preprocessor config is deliberately conservative. Set video_preprocessor_config.json to {"longest_edge": 469762048, "shortest_edge": 4096}, which gives about 224k video tokens. Per-request fps control works only in vLLM, and only when it is launched with --media-io-kwargs '{"video": {"num_frames": -1}}'; the default is fps=2.

Limits

• YaRN is static in all open frameworks. The scaling applies regardless of input length and can hurt short-input quality, so enable it only when you need long context.
• presence_penalty has a cost. Values toward 2 reduce repetition but can cause language mixing and a slight quality drop.
• Low reasoning effort can backfire. In multi-turn agent tasks, low may raise total latency and token use through more failures and retries.
• Benchmarks are hard to compare. Much of the evidence is in-house or on author-corrected benchmarks, and the prompting differed between Qwen and competitors. The fetched page also omits the actual scores.
• Licence is unconfirmed. Apache-2.0 appears only in the catalog title, not in the fetched card text, so check it before adopting.