Tencent Hy4 preview (open-weight MoE + Sherry quantization)

What it claims

Hy4 preview is Tencent Hy Team's 770B-total / 49B-active MoE flagship. Tencent says it reaches the open-source frontier and shows "the largest generation-over-generation gain we've measured" over Hy3 preview. Its training data was built with in-house engineers, game developers, finance analysts and security experts. The target workloads are long-horizon software engineering, turning multi-file context into documents, spreadsheets and presentations, prompt-to-playable game prototypes, and research reasoning. The model is co-designed with Tencent's CodeBuddy and WorkBuddy products.

How it works

The architecture adds two details the summary omits:

• First layer: layer 1 is a standard dense FFN. Only the remaining 77 layers use MoE.
• MTP layer: the speculative-decoding layer has 0.7B activated parameters out of its 10B. The published spec table excludes it.

Tencent credits DeepSeek and GLM as the inspiration for Gated DSA and IndexCache. The page links the IndexCache and DeepSeek-V3.2 papers but does not explain either mechanism itself.

Evidence

The blind side-by-side evaluation had 163 internal experts rating outputs on 203 engineering tasks:

Tencent itself calls these margins "slightly ahead." Both the raters and the tasks are internal to Tencent.

How to use it

Weights for both tencent/Hy4-preview and tencent/Hy4-preview-FP8 are under Apache 2.0, on Hugging Face, ModelScope, GitCode and CNB. The official recipes serve the FP8 checkpoint with 8-way tensor parallelism and 3-token MTP speculation.

The vLLM recipe uses the dedicated image and the sparse-MLA attention backend:

docker run --gpus all \
  -p 8000:8000 \
  --ipc=host \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:hy4-preview tencent/Hy4-preview-FP8 \
    --tensor-parallel-size 8 \
    --speculative-config '{"num_speculative_tokens":3,"method":"mtp"}' \
    --attention-backend FLASHMLA_SPARSE \
    --tool-call-parser hy_v4 \
    --reasoning-parser hy_v4 \
    --enable-auto-tool-choice \
    --port 8000 \
    --served-model-name hy4-preview

The SGLang image is multi-arch (x86 and Arm):

docker pull lmsysorg/sglang:hy4-preview
docker run --gpus all --ipc=host -p 8000:8000 lmsysorg/sglang:hy4-preview \
  python3 -m sglang.launch_server \
    --model tencent/Hy4-preview-FP8 \
    --tp-size 8 \
    --reasoning-parser auto \
    --tool-call-parser auto \
    --speculative-algorithm NEXTN \
    --speculative-num-steps 3 \
    --speculative-eagle-topk 1 \
    --speculative-num-draft-tokens 4 \
    --port 8000 \
    --served-model-name hy4-preview

The API is OpenAI-compatible. Recommended sampling is temperature=0.9, top_p=1.0. Reasoning effort defaults to "high" (deep chain-of-thought). To get direct answers, disable it per request:

extra_body={"chat_template_kwargs": {"reasoning_effort": "no_think"}}

The page also points to a separate finetuning guide and to AngelSlim for quantization and compression.

Limits

• Known issues: Tencent calls this an early version. It reasons longer than necessary on complex tasks and tends to over-verify its own work, which is a latency and token-cost concern with the default "high" reasoning mode. Tencent says it will keep iterating quickly.
• Benchmarks without baselines: the model card lists self-reported scores (GPQA Diamond, Deep SWE, Terminal-Bench 2.1, Toolathlon and others) with no comparison models or evaluation settings. Several entries, including SWE-Bench Pro, have no number shown.
• "Sherry quantization" is not described: the page never mentions the method named in the catalog title. Quantization appears only as a pointer to AngelSlim, so check the AngelSlim repository before assuming a Sherry method exists.
• Missing details: the page gives no context length, training compute or hardware memory requirements. The only hardware hint is the 8-way tensor-parallel FP8 recipe.