EmbeddingGemma 2 Explained: On-Device Multimodal Search

10 min read

Came here from Instagram, LinkedIn, X or YouTube? Here is the full source list you came for. Everything below is built from these, nothing else.

The announcement

• Bring multimodal semantic search to the edge with EmbeddingGemma 2, Google Developers Blog, Oct 6 2026

Picture a map of your phone. Not a folder tree. A map.

Every photo, every video moment, every voice memo and every sentence you type gets dropped on it as a pin. The rule for where a pin lands is simple: things that mean the same thing land close together. A photo of a cat asleep on a laptop sits right next to the typed words "cat sleeping on keyboard". A clip of a dog jumping for a frisbee sits near the phrase "dog catching a frisbee", no matter what language you typed it in.

Search on that map is just one question: which pins are closest to the one I just dropped?

That map is what an embedding model builds. EmbeddingGemma 2, which Google DeepMind launched on October 6, 2026, builds one map for text, images, video frames and audio at the same time, and it is small enough to build it on your phone. That last part is why you should care.

Key takeaways

• EmbeddingGemma 2 is an open-weight multimodal embedding model with 740M parameters that puts text, images, video frames and audio into one vector space.
• It runs in about 191MB of active RAM for text-only weights and about 567MB for the full multimodal model on a Google Pixel 11 Pro.
• Vectors come out at 768 dimensions, or truncated to 128 to 512, and that truncation can cut storage and index size by up to 8x.
• Image embeddings take as little as 37.3 ms (26.9 images per second) on a MacBook M5 Pro GPU, and on-device vector search returns ranked matches in single-digit milliseconds.
• The MediaPipe Decision Task evaluated 500 options in under 100ms per turn in a chess demo, with no training data or fine-tuning.
• You can build with MediaPipe Tasks (iOS, macOS, Windows, Linux, Web) or LiteRT for fine-grained control. ML Kit on Android is coming in the following weeks.

What is EmbeddingGemma 2, in plain words?

An embedding model turns a piece of content into a list of numbers, called a vector. The vector is the pin's coordinates on the map.

Most embedding models only understand one kind of content. A text model places sentences. An image model places pictures. If you wanted to search photos with words, the usual trick was to chain models together: caption the image with one model, transcribe the audio with another, then embed the resulting text with a third.

Google's announcement names that exact chain, image captioning plus speech-to-text plus text embedding, as the thing EmbeddingGemma 2 removes. Instead of translating everything into text first, it maps each modality straight onto the same map. Fewer models means less latency and less memory, which matters a lot when the "server" is a phone.

Three properties define it:

1. Multimodal. Text, images, video frames and audio share one vector space.
2. Open-weight. You can download it and run it yourself.
3. Built for local, privacy-first apps. The content and the search never have to leave the device.

How does on-device multimodal search work, step by step?

Google shipped a demo called Instant Media Search inside its Google AI Edge Gallery app. It is the cleanest way to see the moving parts, so let me walk through it.

Step 1: Index your media once

Each image or video on the device goes through EmbeddingGemma 2 and comes out as a vector. In the demo, those vectors are stored in a local SQLite database. No upload. No internet connection required.

Step 2: Embed the query as the user types

When you type a search, the query goes through the same model and becomes a vector too. Because it is the same model, a sentence and a photo land on the same map and can be compared directly.

The query does not have to be text. In the demo you can also pick an image from your phone, or point the live camera at something, and search with that.

Step 3: Rank by closeness

The app compares the query vector against the stored vectors using cosine similarity and returns the ones with the largest score. Highest similarity means closest pin.

Step 4: Repeat on every keystroke

This is the part that feels different. Embedding the query and ranking the results is fast enough to run on every keystroke, so results update live.

Google's example: type "Katze" (German for "cat") and the grid fills with every cat photo in your collection. Keep typing until you have "Katze schläft auf Tastatur" ("cat sleeping on keyboard") and the specific photo you had in mind moves to the top. That is a multilingual text query matched against images, with nothing in between.

The same idea, applied to video

A second demo, Video Moments Finder, does this for video. You pick a local video, the engine indexes vision together with audio chunks and generates embeddings for them. Then you type "kids laughing" or "person blowing out birthday candles", and it highlights the timestamps where those moments happen.

No audio transcription. No generated captions. Just vectors compared to vectors.

What makes a 740M model fit on a phone?

Google lists three optimizations under the hood. Each one is a dial you can actually turn.

Optimization	What it does	Why you care
Modular encoders	Load only the encoders you need, for example text plus vision for photo search, and add audio when voice memos show up	You pay RAM only for the modalities you use (about 191MB text-only vs about 567MB full, on a Pixel 11 Pro)
Quantization-Aware Training	Compresses weights to INT4 and INT8	Puts multimodal search within reach of mid-range hardware
Adaptive input and output scaling	Loads the right compute graph for the input size at runtime, lets you set a vision token budget, and slices output dimensions with Matryoshka Representation Learning	You trade latency against precision, and cut storage and index size by up to 8x

That Matryoshka slicing is worth a second look. The full vector is 768 numbers. You can cut it down to anywhere from 128 to 512 on the fly. Shorter vectors mean a smaller index on disk and faster comparisons. You give up some precision in return, and you decide where that line sits.

The vision token budget works the same way. Google's published image latency numbers were measured with a maximum of 70 vision tokens per image. A lower budget is faster. A higher one sees more detail.

Which tool should you build with: MediaPipe, LiteRT or ML Kit?

There are three paths, and they stack on top of each other. LiteRT is the engine underneath ML Kit, MediaPipe Tasks, the Gallery app and Foresight.

MediaPipe Tasks, if you want it done for you. It covers iOS, macOS, Windows, Linux and Web with one developer experience. Two tasks matter here:

• Universal Embedder handles image resizing, tensor normalization and multimodal tokenization. You pass raw images or text strings and get back normalized 768-dimension vectors, or truncated 128 to 512.
• Semantic Retriever is an embedded vector search engine. It indexes embeddings on-device and runs Approximate Nearest Neighbour searches, returning ranked matches in single-digit milliseconds.

LiteRT, if you want control. Use it when you need fine-grained control over processing and acceleration. It runs across CPUs, GPUs and NPUs, and a single .litertlm file runs on both CPU and GPU across all supported edge platforms, including the web through WASM. Pre-quantized .litertlm bundles are on the Hugging Face LiteRT Community.

ML Kit, if you ship on Android and can wait a few weeks. Google says it is bringing EmbeddingGemma 2 to ML Kit in the coming weeks, with NPU acceleration where available, automatic model updates, and no model management or APK bloat for you to deal with.

My rule of thumb: start on MediaPipe Tasks, drop to LiteRT only when a specific latency or memory number forces you to.

What else can an embedding model do besides search?

This is the use case I did not expect. EmbeddingGemma 2 can act as a decision engine.

The trick is to embed your labels instead of your content. Write out the options your app can take, with short descriptions, and embed each one. When the user does something, embed their input and pick the closest label. That is classification with no training data and no fine-tuning, which Google calls zero-shot intent routing.

Through the MediaPipe Decision Task, this works on image, text or audio inputs. In Google's chess demo, the Decision Task evaluated 500 options each turn in under 100ms, fully on-device.

Think about what that replaces. "Is this voice command asking for the camera, the timer or the music player?" used to mean training a classifier or calling a large model over the network. Here it is a vector comparison against a short list you wrote by hand.

When should you use EmbeddingGemma 2, and when not?

Use it when:

• The data is personal. Photos, meeting transcripts, voice notes. Google's whole framing is privacy-first: the content stays on the device.
• You need to search across formats. Text finding images, an image finding similar images, words finding a moment in a video.
• It has to work offline. The Gallery demos run with no internet connection, and Google's Foresight app for Mac works with any meeting platform, even completely offline.
• Latency is the feature. Search-as-you-type only feels right when each update is near instant, which is where on-device ranking in single-digit milliseconds helps.
• You need cheap intent routing. Embedding a list of labels is faster to set up than training a classifier.

Think twice when:

• Your target devices are tight on memory. About 567MB for the full multimodal model is real weight on a low-end phone. Load the text-only encoder, or text plus vision, if that is all you need.
• You need a managed Android path today. ML Kit is announced, not shipped. Until it lands, you manage the model yourself through MediaPipe or LiteRT.
• You want an app to depend on, not a demo. Foresight for Mac is labelled experimental. Treat it as a reference, not a dependency.
• The real job is answering questions, not finding things. An embedding model ranks content. It does not write answers. Foresight pairs EmbeddingGemma 2 with Gemma 4 models for the parts that need generation.

A small example: a private photo search, drawn in words

Here is the whole flow for a photo search app, written as a diagram in words. Each box maps to a piece Google described.

INDEX (runs once per new photo)
  photo.jpg
    -> Universal Embedder        resizes, normalizes, embeds
    -> vector [768 numbers]      or sliced to 128-512 with MRL
    -> Semantic Retriever index  stored on the device

QUERY (runs on every keystroke)
  "Katze schläft auf Tastatur"
    -> Universal Embedder        same model, same vector space
    -> query vector
    -> Semantic Retriever        ANN search, single-digit ms
    -> top matches by similarity
    -> photo grid re-ranks live

ROUTE (optional, no training)
  labels: "open camera", "set timer", "play music"
    -> embed each label once
  user input (text, image or audio)
    -> embed, pick closest label
    -> app takes that action

And here is the "closest pin" step in plain Python, so the math is not a black box. Because the vectors come out normalized, cosine similarity is just a dot product.

import numpy as np

def top_k(query_vec, index_vecs, k=5):
    # vectors are already normalized, so dot product = cosine similarity
    scores = index_vecs @ query_vec
    best = np.argsort(-scores)[:k]
    return best, scores[best]

In a real app you would not write this loop yourself. The Semantic Retriever does it with Approximate Nearest Neighbour search, which is built to stay fast as the index grows. But this is all "semantic search" means under the hood: embed, compare, sort.

How do you try it today?

You can see it working before you write any code:

1. Install Google AI Edge Gallery from the Google Play Store or Apple App Store and try Instant Media Search and Video Moments Finder on your own photos.
2. On a Mac, download Google AI Edge Foresight to see retrieval across meeting transcripts, notes, documents and images.
3. Grab pre-quantized .litertlm bundles from the Hugging Face LiteRT Community.
4. Benchmark on real hardware through Google Cloud's Developer Device Platform, which Google says covers over 200 physical devices, or start from the interactive Google Colab.

FAQ

What is EmbeddingGemma 2?

EmbeddingGemma 2 is an open-weight multimodal embedding model from Google DeepMind, launched on October 6, 2026. It has 740M parameters and maps text, images, video frames and audio into a single vector space. It is designed for local, privacy-first apps like on-device search and media retrieval.

How much memory does EmbeddingGemma 2 need on a phone?

On a Google Pixel 11 Pro, Google reports about 191MB of active RAM for the text-only weights and about 567MB for the full multimodal model. Because the encoders are modular, you only load the modalities your app actually uses. Weights are compressed to INT4 and INT8 through quantization-aware training.

Does EmbeddingGemma 2 need an internet connection?

No. The Instant Media Search demo computes embeddings on the phone with no internet connection required, and stores vectors in a local SQLite database. Google's Foresight app for Mac also works completely offline.

Can I use EmbeddingGemma 2 in a web app?

Yes. MediaPipe Tasks supports Web alongside iOS, macOS, Windows and Linux, and LiteRT runs on the web through WASM. Google also points to a zero-setup web demo as a browser-based playground.

Pick ten photos on your phone, run them through the Gallery demo, and search them in two languages before you write a line of code. Then grab the free tools on Art of Code for the boring parts of your build, and if on-device AI is the work you want to do full time, keep an eye on developer jobs on Art of Code.

Written by Adithya Guttha, Founder of Art of Code.