Google Gemini Embedding 2 for multimodal RAG
A Google Developers post about one embedding model for text, images, video, audio, and documents, which is very relevant for agentic RAG.
Most RAG pipelines treat retrieval as text-only, but real company data is screenshots, PDFs, product images, and audio recordings mixed together. One embedding space for all of it changes what retrieval can do.
Source
Building with Gemini Embedding 2: Agentic multimodal RAG and beyond
By: Google for Developers
Published: Apr 30 2026
Type: Engineering article
This article matters because it treats retrieval as multimodal from the start instead of bolting image search onto a text pipeline. That’s a different design assumption, and it opens patterns most RAG examples don’t cover.
How one embedding space for everything changes retrieval design
Gemini Embedding 2 maps text, images, video, audio, and documents into a single embedding space, supporting over 100 languages. A single API call handles up to 8,192 text tokens, 6 images, 120 seconds of video, 180 seconds of audio, and 6 pages of PDFs. The key capability is interleaved inputs — you can embed a combination of text and images in one request, producing a single vector that captures both. That’s different from embedding each modality separately and hoping the cosine similarities align.
For retrieval, task prefixes make a meaningful difference. You prepend a task description to both the query and the document at index time: task: question answering | query: ... for asymmetric retrieval (short queries matching long documents), or task: clustering | query: ... for symmetric tasks where query and document have similar length. The prefix tells the model what kind of match to optimise for, bridging the gap between different input shapes. Different prefixes exist for question answering, fact checking, code retrieval, search results, clustering, classification, and sentence similarity.
The practical use cases in the article are concrete. Harvey, a legal research platform, saw a 3% increase in Recall@20 precision on legal-specific benchmarks compared to their previous embeddings — that 3% translates directly into more accurate citations for law firms. Supermemory, which builds a “vector database for memory” for searching across disjointed memos, achieved a 40% increase in search Recall@1 accuracy. Nuuly, a clothing rental company, uses the model for visual search that matches photos of untagged warehouse garments against their catalogue — Match@20 accuracy went from 60% to nearly 87%, and total successful product identification from 74% to over 90%.
For storage efficiency, the model is trained using Matryoshka Representation Learning (MRL), so you can truncate the default 3,072-dimensional vectors to smaller dimensions (1,536 or 768 recommended) using an output_dimensionality parameter. Lower dimensions mean lower storage costs while maintaining high accuracy. The Batch API adds another cost lever: higher throughput at 50% of the default embedding price.
A useful technique the article describes is search reranking with hypothetical answers. Prompt the model to generate a baseline answer to a query using its internal knowledge, embed that template, then compare its similarity score against your retrieved data to rank the most accurate match. This works because the hypothetical answer is closer in embedding space to the right document than the original short query is.
The edge case to watch is interleaved input quality. When you embed a mix of text and images in one request, you get one aggregated vector. If the text and image say different things, the vector captures some blend of both — and the retrieval behaviour depends on which modality dominates. For cases where you need separate control, the Batch API can produce individual embeddings per input instead.
My take
The real shift here isn’t that multimodal embeddings exist — it’s that they’re in the same API as text embeddings with the same interface. That removes the architectural split between “text RAG” and “image search” that most systems have today. The practical implication is that agents working over real company data — where a single document might be a PDF with text, charts, and photos — can now retrieve across all of that with one query. The question is whether the unified embedding space actually captures cross-modal relationships well enough for production. The case studies suggest it does for specific domains (legal, warehouse inventory, memory search), but the gap between “works on our benchmark” and “works on your messy document store” is still worth testing.
Where this is useful
- When building RAG over mixed company data (PDFs, screenshots, images, audio recordings) — one embedding model for all modalities eliminates the need to maintain separate retrieval pipelines per content type.
- When retrieval accuracy matters more than raw speed — task prefixes and search reranking with hypothetical answers are concrete techniques that improve results without changing the architecture.
- When storage costs for embeddings start to matter at scale — MRL-based dimension truncation (3,072 → 768) and the Batch API pricing are practical cost levers.
Apply this: If your RAG pipeline is text-only but your actual knowledge base includes screenshots, PDFs, or images, test embedding those into the same space. The retrieval improvement on real queries is usually larger than you’d expect.
What I want to remember
Remember: Future RAG systems shouldn’t assume all important context is text. One embedding space for text, images, video, audio, and documents means retrieval can match how real data actually looks — mixed, messy, and multimodal.