Granite 4.2 LLMs: How They're Built

What it claims

Granite 4.2 (3B, 8B, 30B, Apache 2.0) adds explicit reasoning to Granite. Each model has three modes: thinking, non-thinking, and a low-effort thinking mode that spends a short reasoning budget. All three sizes do native tool calling. Their capabilities differ in post-training. Only the 8B and 30B models get agentic RL in real sandboxes (SWE, terminal, web search). The 3B model gets foundational RL plus alignment. If you want a local coding agent, the 3B is not trained for it.

How it works

Architecture. The page gives one set of values, not a breakdown per size:

• Dense decoder with GQA, 40 query heads and 8 KV heads
• RoPE with θ = 10,000,000
• SwiGLU MLP and RMSNorm (ε = 1e-5)
• Untied input and output embeddings
• bf16

SFT. About 7.2M samples, roughly 100B tokens, of which about 65B are trainable.

• The mix is 31.6% agentic and 68.4% non-agentic.
• Within the agentic part, 69% is SWE. It was generated across about a dozen harnesses, including OpenHands, SWE-agent, Codex, Gemini CLI and Goose.
• Filtering removes samples with low judge scores, hallucinated content, invalid tool interactions, or calls to functions missing from the tool list.
• Deduplication uses a SHA-256 hash over the tools and messages fields together.
• The 30B gets a second SFT phase of about one epoch at learning rate 3.0e-6. It upsamples agentic and coding data and keeps about 16% replay from the first SFT corpus.

RL. Each stage is a separate asynchronous GRPO run, warm-started from the previous stage's checkpoint.

• Advantages use a leave-one-out group baseline, so there is no value network.
• Generator workers may be at most one policy update behind the trainer.
• A trajectory can mix two adjacent policy versions, because workers keep their KV cache across weight refreshes.
• Truncated importance sampling clamps the ratio between train and generation log-probabilities.
• RLVR uses 256 prompts × 16 samples, a 4,096-example batch per optimizer step. It runs 2 rounds on 3B and 8B, and 3 on 30B.
• KL penalty follows the reward type: 0 for verifiable stages (RLVR, an SWE stage), 0.05 for the code booster and RLHF.
• RLHF also adds a penalty on reasoning length.

flowchart LR
  SFT --> RLVR --> B[Skill boosters]
  B -->|8B / 30B| SWE[SWE agent] --> T[Terminal] --> S[Search]
  B -->|3B skips agentic| RLHF
  S --> RLHF

Agentic stage rewards:

• SWE: hidden tests pass, run through OpenHands with one container per repository.
• Terminal: task success via Harbor/Terminus-2, up to 64 environment turns. This is the only stage whose agent loop runs at the GRPO level.
• Search: an LLM judge scores the final answer.

Infrastructure. Training used NeMo-RL (Megatron-Core, vLLM for rollouts, Megatron-Bridge for HF conversion) and NeMo-Gym for environments, on CoreWeave GB200 NVL72.

How to use it

pip install torch
pip install accelerate transformers

Model and sampling. The model ID is ibm-granite/granite-4.2-30b. The examples sample at temperature=1.0, top_p=0.95.

Chat-template flags:

• enable_thinking=True/False. Non-thinking mode still emits an empty <think></think>.
• low_effort=True
• truncate_history_thinking. Earlier turns' reasoning is stripped by default.

Tool calls. In raw Transformers output, tool calls use an XML-like format (<tool_call><function=...><parameter=...>). When served through vLLM's OpenAI-compatible endpoint, they come back in OpenAI function-calling format. SGLang is also supported.

Agent harness configs. The page gives configs for OpenCode, Pi and OpenHands, all pointing at http://localhost:8000/v1:

• OpenCode and Pi set context to 131072 and output to 8192.
• Pi needs supportsDeveloperRole: false and supportsReasoningEffort: false.
• OpenHands needs the openai/ model prefix.

Quantized variants:

• FP8: dynamic, per-channel weights and per-token activations, no calibration.
• NVFP4 and MXFP4: GPTQ calibrated on 2K SFT samples at 2K context.
• GGUF: Q8_0 down to Q2_K.

Languages: English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch and Chinese.

Limits

• No benchmark numbers in the text. The fetched text has none; results are only in tables and charts that were not captured. The page says only that scores rise with size, and that the 30B leads on the SWE-Bench variants and Terminal-Bench.
• No external baselines are named in the text.
• Context mismatch. Pretraining extends context to 512K, but the harness configs use 131072. The page does not explain the gap.
• Calibration context. FP4 calibration used only 2K-token context.
• Judge-based rewards. Search reward and data filtering both rely on LLM judges.
• Pretraining details are deferred to the Granite 4.1 blog.