google/embeddinggemma-2

What it claims

EmbeddingGemma 2 is a single embedding model that maps text, code, images, video and audio into one 768-d space. Google DeepMind claims three things for it:

• It understands 100+ languages.
• It scores about 14% better on code tasks than its predecessor.
• Matryoshka truncation cuts vector storage by up to 6x, with quality "close to lossless" down to 256d.

The card also calls it "among the strongest multimodal embedding models under 1B parameters."

How it works

• Text backbone (270M): a 130M-parameter transformer plus a 140M-parameter embedder. Most of the text model is the embedding layer, so shrinking the transformer would save little.
• Optional encoders: the vision (170M) and audio (300M) encoders are independent. You can leave them unloaded for text-only or single-modality pipelines. The config_kwargs example is not in the fetched text, and the card says the loading mechanism differs by library.
• Shared context: text and media share one 8,192-token context. Placeholder tokens <|image|>, <|video|> and <|audio|> mark where each media item goes in the text. Each placeholder is filled in order from the matching input key.
• Vision token budget: images and video frames can use 70 to 1,120 "soft tokens" each. More tokens means better quality and higher latency. At a low budget, up to about 114 images or frames fit in one input.
• Input defaults: video is sampled at 1 fps (configurable). Audio must be mono at 16 kHz.
• Training data: web text (140+ languages), code, images, video, audio and paired cross-modal samples, with a cutoff of January 2025.

How to use it

pip install -U sentence-transformers transformers

query_emb = model.encode(query, prompt_name="SearchQuery")
doc_emb = model.encode(document, prompt_name="Document")

• Prompt names: the card names SearchQuery and Document.
• prompt_name="Document" inserts title: none. If a document has a real title, format it yourself as f"title: {title} | text: {document}".
• Retrieval is asymmetric: use the query prefix on queries and the document prefix on documents.
• Classification and similarity are symmetric: use the same prefix on every input.
• Prefixes apply to text only. Pass media without one.
• Truncation in one call:

  model.encode(query, truncate_dim=128, normalize_embeddings=True)

• Precision: use bfloat16 or float32.

  dtype = torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float32
  model = SentenceTransformer("google/embeddinggemma-2", model_kwargs={"torch_dtype": dtype})

Use float32 on most CPUs.

• Interleaved input returns one embedding that can be compared against text-only queries:

  model.encode({"text": "Waterproof running shoes. <|image|> ... <|video|>",
                "image": ["shoe.jpg", "mesh.jpg"], "video": "demo.mp4"})

Limits

• 128d is not safe for multimodal use. The card says it "degrades multimodal quality substantially" and recommends it mainly for text-only workloads. Validate it on your own data first.
• Silent failures:
• float16 produces NaN or degraded embeddings without raising an error.
• Forgetting to re-normalize after manual truncation gives plausible-looking scores with worse ranking.
• Queries and the corpus must use the same dimension.
• Benchmarks not available here. The benchmark and truncation tables, and the per-modality context consumption table, did not come through in the fetched text. The "strongest under 1B" claim and the 14% code figure therefore have no visible numbers or baselines.
• Uneven language quality: the card says performance may not be equal across languages.
• No safety tuning: the model has no post-training alignment or safety tuning. Mitigation was limited to pre-training data filtering, and downstream safeguards are the deployer's job.
• Licence: Apache 2.0, but deployments must also follow the Gemma Prohibited Use Policy.