Qdrant Shrinks Perplexity's Visual Retriever Vectors 21x With TurboQuant

Searching page images instead of extracted text

Perplexity released pplx-embed-v2-late, two late interaction models that search document pages as images. The model embeds the page image, including its tables, charts and layout, so developers can skip the optical character recognition (OCR) step and search the page as it appears. Qdrant argues that this context matters: a number in a financial report needs its row, column and unit to mean anything, and a chart needs its labels.

The catch, according to Qdrant, is the size of the representation. Late interaction models keep one vector per token, and pplx-embed-v2-late turns each page image into about 1,700 vectors of 128 dimensions. MaxSim scores a page by finding the best document-vector match for each query token and summing those matches. In float32 that comes to 856 KB per page, so a million pages would need over 800 GB for their vectors alone.

What TurboQuant does to the numbers

Qdrant's TurboQuant stores each vector in a few bits. At 1 bit, the compressed vectors take 40 KB per page instead of 856 KB, without removing tokens or converting the page to text. In Qdrant's tests, searching the compressed vectors retained 96.5% of the retrieval quality of float32. Rescoring, which checks the top candidates against the original vectors, raises that to 99.9%. Rescoring uses four times as many candidates.

At the same 1-bit setting without rescoring, binary quantization kept 76.4% of retrieval quality, against 96.5% for TurboQuant. Other settings from the 9b model on page images:

• 4-bit TurboQuant: 120 KB per page, 99.4% without rescoring, 100% with it.
• 2-bit TurboQuant: 67 KB per page, 98.5% and 99.9%.
• 2-bit binary: 54 KB per page, 95.6% and 99.9%.
• 1-bit binary: 27 KB per page, 76.4% and 99.4%.

For a million pages, the compressed representation would be about 40 GB. Qdrant notes that it also stores the originals, and suggests keeping compressed vectors in RAM and originals on disk to reduce memory use. The million-page figures are projections from per-page sizes, not a million-page search benchmark.

The tests used four public ViDoRe v3 sets (HR, finance, industrial and pharmaceuticals, 11,609 pages in total) with 200 English queries each. Quality was measured with nDCG@10 and reported as the share of the exact float32 score retained, averaged over the four sets.

Big index model, small query model

Because a document embedding can be reused across searches while every query needs its own, Qdrant says it makes sense to spend more compute on indexing than on each query. Perplexity's two models share one embedding space, so pages can be indexed once with the 9b and queries embedded with the 0.6b, which has about 20x fewer active parameters.

On page images, this pairing scored 0.670 nDCG@10, between 0.666 for the 0.6b alone and 0.681 for the 9b. On page Markdown the scores were 0.653, 0.658 and 0.671, and on FiQA text retrieval 0.522, 0.588 and 0.622. Qdrant says the gain over the smaller model alone was modest on page images and larger on FiQA. With 1-bit TurboQuant, the mixed setup kept 96.6% without rescoring and 100% with it, compared with the same pairing in float32.

What it means for OCR and for developers

Qdrant says it thinks OCR should become optional for document retrieval, and that for visually rich PDFs it would test searching page images before building a text-extraction pipeline. OCR is still needed if an application requires extracted text or structured fields. In the tests, page images scored 0.681 nDCG@10 with the 9b, against 0.671 for the same pages converted to Markdown. A Markdown page took 14 KB of compressed vectors, compared with 40 KB for a page image.

Qdrant's advice is to keep the page image when layout matters and to test Markdown when text carries the information. Teams can set up a multivector collection with TurboQuant at 1 bit, compare compressed search against an unquantized baseline on their own queries with rescoring explicitly disabled, then turn rescoring on and check whether the quality gain is worth the extra reads from the original vectors.