Welcome Le Chonk: Mistral Large 4 on Hugging Face

What it claims

Mistral Large 4 ("Le Chonk") is an image-and-text-to-text MoE model with about 1T total parameters. The repository ID is mistralai/Mistral-Large-4-1T-A52B, although the prose says 49B active. It has day-0 support in transformers, vLLM, llama.cpp, TRL and Inference Endpoints. The licence is the Mistral Ecosystem Open-Weight License, described as an MIT derivative that limits model-as-a-service offerings. Official variants include an FP8 checkpoint and an NVFP4 checkpoint (-NVFP4). GGUFs come from Unsloth and the llama.cpp team. Hugging Face reports it as the fifth-best open-weights model on the Artificial Analysis Intelligence Index as of October 29, 2026.

How it works

The text decoder has 72 layers, hidden size 6,144 and 64 attention heads. Attention combines Multi-head Latent Attention, which stores a compressed KV, with a DeepSeek Sparse Attention indexer that selects up to 2,048 key positions per query. Layers follow a 3:1 RoPE-to-NoPE pattern, and KV state is shared across layers. Positional encoding is "LeRope", a YaRN-scaled RoPE with one learned multiplier per frequency.

Layer 1 uses a dense FFN. The other 71 layers are MoE: a softmax router picks 4 of 128 experts, and one shared expert always runs, so 5 experts are active per token. The vision side is a 1.6B Pixtral encoder with 30 layers, hidden size 2,048, 16 heads, 14×14 patches and a patch-merge projector with spatial merge size 2. The checkpoint includes a "DSpark" draft head for speculative decoding.

Evidence

The page reports these coding results: DeepSWE v1.1 61.7%, SWE-Atlas-QnA 59.4%, Terminal-Bench 4 28.3%, and a Coding Agent Index of 49.8%, ahead of DeepSeek V4 Pro 0813 and Qwen3.8 Max. Knowledge-work results are AutomationBench 59.9% and AA-Briefcase 1,393 Elo.

For security, the model scores 82% on the Artificial Analysis Cyber Index reproduce-then-patch task, the highest score in that index. It ranks top five overall on the index and solves 93% of Cybench's 40 challenges. Mistral says several closed frontier models score near zero on the reproduce-then-patch task because they refuse it. The model reaches 93.3% attack resistance on Lakera's B3 prompt-injection benchmark. Its average refusal rate on the cyber prompts of JailbreakBench, StrongREJECT and AgentHarm is higher than any other open-weight model's. Mistral's internal claims of usefulness for malware analysis and detection rules come without numbers.

Measured serving figures:

• Inference Endpoints, 8×H200, FP8, vLLM with DSpark: TTFT about 0.6 s for a 1K-token prompt. Generation is about 65 tok/s, rising to 120–138 tok/s when drafts are accepted. Four concurrent 16K-token prompts prefill at about 4,300 tok/s total, with 8–9 s TTFT. Total throughput is about 480 tok/s across 1,000 ShareGPT conversations.
• llama.cpp Q2_K on one M3 Ultra (512 GB): about 90 tok/s prefill and 15 tok/s generation.
• 4×RTX6000 Pro: NVFP4 at 32K context runs at 130–590 tok/s prefill and about 10 tok/s generation. IQ2_XXS reaches 800K context, with 500/30 tok/s at short context and 129/3 tok/s at long context.

How to use it

Transformers needs a distributed launch (torchrun --nproc-per-node 8 script.py) with AutoModelForMultimodalLM and DistributedConfig(tp_size=..., ep_size=...). vLLM serves the NVFP4 checkpoint, which requires Blackwell GPUs:

vllm serve mistralai/Mistral-Large-4-1T-A52B-NVFP4 \
  --config-format mistral \
  --load-format mistral \
  --tensor-parallel-size 8 \
  --decode-context-parallel-size 8 \
  --enable-expert-parallel \
  --dtype bfloat16 \
  --max-model-len 131072 \
  --max-num-seqs 16 \
  --max-num-batched-tokens 2048 \
  --gpu-memory-utilization 0.85 \
  --enable-prefix-caching \
  --enforce-eager \
  --mm_processor_cache_gb 0.0

For llama.cpp, the quants are IQ2_XXS, Q2_K, Q4_K_M, NVFP4, Q8_0 and BF16. Only the 2-bit quants fit on a single 512 GB Apple Silicon Ultra; the others need distributed inference.

llama serve -hf ggml-org/Mistral-Large-4-1T-A52B-GGUF:Q2_K

Inference Endpoints deploys with one command:

hf endpoints catalog deploy --repo mistralai/Mistral-Large-4-1T-A52B --name mistral-large-4

The endpoint returns the answer in choices[0].message.content and the reasoning in choices[0].message.reasoning. It supports tool calls, structured outputs, images and streaming. A Dell Enterprise Hub recipe deploys the NVFP4 checkpoint on 4 GPUs of a B300 node.

The TRL example runs LoRA SFT (r=16, alpha 32) on the FP8 checkpoint using 8×B300. It uses FSDP plus expert parallelism, packing=True, max_length=4096 and assistant_only_loss=True. Training data carry reasoning_effort: "high" so that reasoning turns render correctly.

Limits

Reasoning cannot be turned off. Effort is low, medium or high (the default), and reasoning_effort: none silently runs as high. The licence restricts model-as-a-service use, and the page does not spell out the terms. vLLM's NVFP4 path is Blackwell-only. Long-context throughput on IQ2_XXS falls to 3 tok/s generation.

The page has internal inconsistencies to check before copying recipes. It says 49B active while the repository ID says A52B. The llama.cpp WebUI is given at port 9931 while the curl examples use 8080. The image example sends "model": "muse-spark-open". Mistral's refusal and malware-analysis claims are its own testing, and the page gives no method.