Qdrant Tests UltraBERT-Embed: 32K-Token Vectors Don't Kill Chunking Yet

What UltraBERT-Embed offers and how Qdrant tested it

According to Qdrant, DMIS Lab's UltraBERT-Embed-150M is a 151M-parameter dense embedding model built on UltraBERT, an English encoder trained for 32K-token inputs. Most documents fit in a single pass, and the model returns one 768-dimension vector. That moves the usual cutoff in retrieval pipelines, where chunks that are too big get truncated by the embedding model and chunks that are too small lose context.

Qdrant compared several setups inside its own database: one vector per document, 512-token chunks, both together, and both compressed. It used five tasks from the LongEmbed benchmark, whose documents average 7,600 to 13,800 tokens: SummScreenFD, 2WikiMultihopQA, QMSum, Passkey and Needle. For testing at scale it added MLDR, with 200,000 English Wikipedia articles and 800 questions, each about one article. A document's chunk-based score came from its best chunk.

Chunks win when the answer sits in a few passages

The whole-document vector does read the full 32K tokens. On Passkey, it ranked the right document first 99% of the time across all lengths, and every time at 32K tokens. Even so, chunks scored higher on most other tasks. They won on 2WikiMultihopQA, QMSum and MLDR, where the answer is in a few passages. The whole-document vector won only on SummScreenFD, where the query describes the whole episode.

Needle shows the limit of a single vector. At 256 tokens, the whole-document vector ranked the right essay first 84% of the time. At 32K tokens, that fell to 30%. At the same length, 512-token chunks reached 56%.

Qdrant also tried combining both. One point can hold two named vectors: the whole-document vector and the chunk vectors as a multivector. A hybrid query searches both and fuses the results. This helped a little on SummScreenFD and Needle, by up to 0.02 and 0.03 over chunks alone. On 2WikiMultihopQA, QMSum and MLDR it did worse than chunks alone, most of all on MLDR.

Compression keeps the extra vectors affordable

Chunking means more vectors. MLDR's articles split into 7 chunks on average, giving 1.43 million vectors, or about 4 GB in float32. With 1-bit TurboQuant, each vector shrinks from 3,072 bytes to 104, which is 30x smaller, and about 140 MB for all 200,000 articles.

• Against exact search, the compressed chunk vectors kept 95% of the quality, and 98% with rescoring.
• The uncompressed vectors in the same index kept 99%.
• Rescoring read original vectors from disk in Qdrant's setup, adding 4 to 11 ms per query, or over 100 ms while the disk cache was cold.

A whole-document vector compresses less well. On MLDR it kept 89% of its quality at 1 bit. At 2 bits it scored the same as the uncompressed vector in the same index, so Qdrant suggests starting there or keeping rescoring on.

What it means for retrieval design

Qdrant's conclusion is that chunking is not dead. It would keep chunks as the starting point for detail search, even when the whole document fits in the window, and test 256-token chunks for single facts. Whole-document vectors also cost more to compute: embedding SummScreenFD's transcripts whole took about 3x as long as embedding 512-token chunks on an NVIDIA L4.

The 32K window does open other options. Qdrant suggests searching whole documents to find the right topic, then searching their chunks for the answer. It calls that worth testing, particularly since combining both searches did not consistently help. It adds that quality and rescoring latency still need testing on your own queries.