Qdrant Tests How Far EmbeddingGemma 2 Vectors Can Be Shrunk

The memory problem behind the model

According to Qdrant, Google's EmbeddingGemma 2 is small enough to run on a phone, but the vectors it produces need much more room. At 10 million documents, full-size float32 vectors alone take 30.7 GB of RAM.

Qdrant tested two ways of cutting that cost: shorter vectors and fewer bits per dimension. It compared the results with exact search over full-size float32 vectors on five text retrieval datasets. Quality is reported as the percentage of the baseline's nDCG@10 retained, a score for how well the top 10 results rank relevant documents.

Qdrant notes that its vector RAM figures exclude graph indexes, payloads, and original vectors stored for rescoring. The results cover text retrieval only, and rescoring uses 4x oversampling.

About EmbeddingGemma 2

EmbeddingGemma 2 is an open embedding model built on Gemma 4. It maps text, code, images, video, and audio into one 768-dimensional space. Its text-only path has 270M parameters. Google reports a 14% gain on code retrieval over the first EmbeddingGemma and an 8K-token context window.

The model supports Matryoshka Representation Learning, which lets users keep the first 512, 256, or 128 dimensions without re-embedding their data. Those leading dimensions remain useful on their own. Quantization is the second lever, storing each dimension in fewer bits.

Full-size, 1-bit vectors first

Qdrant's suggested starting point is to keep all 768 dimensions, use 1-bit TurboQuant, and leave rescoring off. Each vector then takes 104 bytes instead of 3,072, which puts the vectors for 10 million documents at about 1 GB of RAM. Skipping rescoring also avoids reading original vectors during search.

On Quora's 523k documents, this setup took less than half the query time of float32 search and retained 99.0% of its retrieval quality. Qdrant cautions that high relevance does not mean identical results: without rescoring, about 74% of the top 10 results matched exact search. Teams that need the same neighbors back can try rescoring.

Qdrant also compared cutting dimensions with cutting bits. At 72 bytes per vector, 512 dimensions at 1 bit kept 97.3% of the baseline. At a similar memory budget, a 128-dimensional, 4-bit setup kept 84.0%. Its advice is to keep more dimensions and compress each one more heavily first.

Going down to 77x less vector RAM

For those counting every byte, Qdrant points to 256 dimensions with 1-bit TurboQuant, at 40 bytes per vector in RAM. With rescoring, this setup retained 94.5% of the baseline, and it reached 77x less vector RAM. Without rescoring, retention dropped to 88.1%.

Qdrant says the relevance loss and rescoring time should be checked against the needs of each application. It also notes that its vector datatypes allow originals to be stored as float16 or turbo4 to save disk space, but it has not tested those options yet.

What it means for users

Qdrant recommends starting with the plain configuration: full-size vectors, 1-bit TurboQuant, rescoring off. Users should compare relevance and latency with an unquantized collection on their own queries, add rescoring if they need better neighbor recovery, and shorten vectors only if they still need to save memory. The figures come from Qdrant's own tests on five text retrieval datasets, so results on other data may differ.