Build specialized multimodal models with LFM3

What it claims

Liquid AI's LFM3 release (2026) says its 1.7B text model beats LFM2.5-2.6B with 35% fewer parameters. The largest gain is on WebArena-Verified: 38.42 against 3.02. The vision model LFM3-VL-2B (1.9B parameters) raises the average across 62 vision benchmarks from 57.6 (LFM2.5-VL-3B, 3.1B) to 71.4, which the post puts on par with Gemma 4 12B. The text model fits in about 1 GB at 4-bit precision. Every model in the family is said to fine-tune on a single GPU.

How it works

The backbone came from Liquid AI's hardware-in-the-loop architecture search. That search covered 14 device and backend combinations, including phones, Raspberry Pi 5, laptops, cloud CPUs, GPUs and the Qualcomm Hexagon NPU. Beyond the layer mix in the summary, the 24-layer base adds four changes:

• Sliding-window attention uses a 256-token window. Convolution kernel size rises from 3 (LFM2) to 7.
• Head-wise gated attention: a sigmoid gate scales each head's output so a head can switch off. Liquid AI says this can suppress activation outliers that hurt quantization.
• Partial RoPE: global layers rotate only the highest-frequency half of each head. SWA layers keep full RoPE.
• Sandwich norm normalizes each block's input and output.

The vocabulary is 128K, shared with LFM2.5-8B-A1B. The last 3.5T of the 22T pre-training tokens are a mid-training stage. It uses curated data with more agentic traces and tool calls, and it extends context to 256K.

flowchart TD
  B[LFM3-1.7B-Base] --> T[LFM3-1.7B text]
  B --> V[LFM3-VL-2B]
  B --> A[LFM3-Audio-2B]
  B --> O[LFM3-Omni-2B]
  B --> D[d1-2B / d1-omni-2B]
  VE[164M vision encoder] --> V
  VE --> O
  AE[Causal audio encoder + LiquidCodec] --> A
  AE --> O

The new vision encoder mixes convolution and attention. It uses 3x less compute than the SigLIP2 encoder in earlier LFM VL models. For video, it packs each pair of frames into one temporal slice. The audio encoder is fully causal, so the model can prefill during speech; time to first token is as low as 50 ms. LiquidCodec decodes speech at about 87x real time on one CPU thread of an Apple M4 Max, with streaming decoding. The d1 models answer a list of typed questions about one input in a single forward pass. Each answer comes with a calibrated probability.

Evidence

• Speed: per parameter, LFM3-1.7B matches LFM2-1.2B on short prompts across edge devices. It is about 25% faster with 8K tokens in context.
• Vision encoder: matches Qwen3.5-2B's encoder in accuracy with 2.2x fewer FLOPs. Both were tested on the same LFM2.5-230M language backbone.
• Audio: UTMOSv2 is 3.83 for LFM3-Audio-2B against 3.22 for LFM2.5-Audio-1.5B. The model covers 18 languages.
• Omni: the post says it beats models up to 10.7B, such as Qwen2.5-Omni-7B, on Daily-Omni, WorldSense and AV-SpeakerBench. No scores are given.
• d1-2B: scores 75.5 against 64.0 for d1-3B on ImajevBench, and is higher on all four decision suites.
• Robotics: a VLA fine-tuned from LFM3-VL-2B was trained across 15 embodiments. It reaches 96.2% average success on the four LIBERO suites and 45.7% on RoboCasa-GR1 tabletop. It produces 8-step action chunks in under 50 ms on a Jetson AGX Orin with TensorRT FP16.

How to use it

All models are open-weight on Hugging Face, and LFM3-1.7B-Base is provided for custom post-training. Each model needs trust_remote_code=True, and the minimum transformers versions differ:

pip install -U transformers
pip install "transformers>=5.21.0" torch torchvision pillow
pip install "transformers>=5.20.2" torch accelerate librosa soundfile
pip install "transformers>=5.20.2" torch torchvision torchcodec accelerate pillow librosa soundfile
pip install "transformers>=5.19" torch torchvision pillow

In order, these are for the text model, VL-2B, Audio-2B, Omni-2B and d1-2B. The suggested sampling settings are temperature=0.1, top_k=50, repetition_penalty=1.1. The VL example sets max_new_tokens=16384 because the model reasons before answering. Audio and Omni use AutoModelForMultimodalLM and decode speech at 24 kHz through processor.audio_tokenizer. d1 is called as model.system_one(text, questions). Question types are noul (yes/no), choice (with multiple: True for multi-select) and score. The text model is post-trained for the Hermes Agent, Pi and OpenClaw harnesses. Model cards cover llama.cpp and other engines.

Limits

The post does not state a licence. Check the model cards before commercial use. The comparisons to Gemma 4 12B, Qwen3.5-9B and Qwen2.5-Omni-7B come without full score tables or evaluation settings. The robotics results are Liquid AI's own fine-tunes, not released checkpoints named in the post. Speed figures are relative ("about 25% faster"), with no absolute tokens-per-second numbers.