Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint

What it claims

Granite-Switch-4.1-8B and -30B are single deployable models. Each bundles a frozen Granite base with three components:

• a general-purpose Decision aLoRA
• a Guardian safety aLoRA
• the Granite-Speech-5.0-Turbo-CTC ASR encoder

IBM claims the Guardian aLoRA, on average, outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions and Clef-Flash on safety-related decision tasks. It also calls Granite-Speech-5.0-Turbo-CTC the fastest ASR model on the OpenASR Leaderboard. The 8B model is said to transcribe audio, summarise, screen for harmful content and test "15+ generic flags" in real time on one endpoint.

How it works

The adapters are Activated LoRAs (aLoRAs), so a decision can reuse the KV cache produced by an earlier base-model generation instead of re-encoding the context. The ASR encoder runs on the same GPU as the base model. IBM's recent vLLM changes for faster LoRA inference handle switching between request types.

flowchart LR
  A[Audio request] --> ASR[Co-located ASR encoder]
  ASR --> B[Granite base]
  T[Text request] --> B
  B --> KV[(Shared KV cache)]
  KV --> C[Chat output]
  KV --> D[Decision aLoRA]
  KV --> G[Guardian aLoRA]
  KV --> S[Specialist granitelib aLoRAs]

The design rests on a cost argument. Adapters share the cache, so stacking specialist adapters next to the generalist one is described as costing "almost nothing." The generalist gives broad coverage and the specialists give higher accuracy on narrow tasks. IBM says its specialised granitelib adapters beat generalist decision models on specialised classification tasks. The chart behind that claim is not in the fetched text.

Evidence

All comparisons are stated qualitatively or without full settings:

• RAG race: Granite-Switch-4.1-30B (frozen Granite base plus 4 specialised aLoRAs) ran against Jev-27B (frozen Qwen base plus 1 generic decision LoRA) on a real-world 5-step RAG workflow. The page says only that Switch finished "much faster." It gives no latency figures, hardware or task description.
• Doom: IBM's custom 3B Switch model reached 60–100 decisions/s. The comparison point is Jev's 10 decisions/s, which comes from Typesafe's own blog rather than a side-by-side run. IBM adds that its model can narrate and answer questions while playing.

How to use it

• Prebuilt models: the 8B and 30B releases are "batteries included" builds with a fixed adapter set.
• Custom builds: you can compose your own Switch model from Granite Libraries adapters.
• Resources listed on the page: model cards, a Speech-to-Decisions demo, the Granite Switch GitHub repo, and a Colab notebook for running your own aLoRA vs LoRA race. The Colab is the most direct way to check the cache-reuse speed claim on your own hardware.
• Not stated: install commands, API shape and licence. Look for these on the model cards.

Limits

• None of the speed or safety claims come with numbers, benchmark names or hardware. IBM says more details and data will follow "in the upcoming weeks."
• The Doom comparison sets IBM's measurement against a competitor's self-reported figure.
• The Adapter SDK for training custom adapters, to ship in Mellea, is announced as planned and not yet released.
• VRAM use is stated only as "approximately the same" as the base model.