Qdrant Tests Jev Classifier on Reranking, Diversity and Query Understanding

Where Jev fits among classifiers

Qdrant's engineers Andrei Cristea and Chadha Sridi describe Jev as a new option in the taxonomy of classification approaches. Before it, teams typically trained their own classifier, which needs labels, used a zero-shot classifier, or prompted an LLM with extra type safety on top.

In Qdrant's comparison, Jev needs no training, accepts new labels at runtime, constrains its output to the given labels and has a typical speed the authors call fast. A generative LLM, by contrast, is described as slow and not constrained to the labels by default. Qdrant says Jev keeps the idea of "describe it, don't train it" while addressing slow responses, high costs and answers outside the available options.

All experiments are published in the many-jev-recipies repository.

Reranking: the cheap method wins

Qdrant tested three ways of asking Jev to reorder search candidates: Score, Choice and Iterative. Score returns numeric scores that sort directly, with nothing to parse. The test used 100 NFCorpus queries, with the top 30 and top 50 candidates from Qdrant using BGE-small. Two local cross-encoders were included for comparison.

• Jev Score (1 request per query): +0.0665 nDCG@10 lift at top 30, +0.0741 at top 50.
• Jev Choice (1 request): +0.0583 and +0.0673.
• Jev Iterative (10 requests): +0.0746 and +0.0789.
• ms-marco-MiniLM-L-6-v2 (local): +0.0313 and +0.0258.
• bge-reranker-base (local): -0.0015 and -0.0127.

The authors expected the costly Iterative method to win by a wide margin. It scored highest, but the gaps between the three Jev methods were under 0.02. Qdrant therefore uses Score as its default and says Iterative is not worth ten requests per query. Both cross-encoders gained less than any Jev method. The authors add that cross-encoders can run locally and may be faster or cheaper, so the right choice depends on flexibility, quality, latency and cost.

Cutting repetitive results

Relevant results can still be repetitive, such as a search for "iphone" returning ten variants of the same phone. Qdrant measured this on 240 held-out queries from the WANDS benchmark, tracking nDCG@10, near-duplicate pairs (embeddings 0.95 similar or more) and the number of product classes in the top 10.

Plain hybrid search scored 0.683 nDCG@10 with 1.27 near-duplicate pairs per page and 3.17 product classes. Jev reranking raised relevance to 0.769 but left 1.25 near-duplicate pairs and cut variety to 2.41 classes. Qdrant's built-in MMR at diversity 0.5 removed duplicates and widened the page to 4.74 classes, but relevance fell to 0.561 because MMR measures relevance as vector similarity to the query.

The fix was to use Jev's score as the relevance signal for diversity. Using it inside the team's own MMR at diversity 0.7 gave 0.719 nDCG@10 and 0.22 duplicate pairs. Running Qdrant's MMR first and then letting Jev rerank the top 20 gave 0.723 and 0.04. Both beat hybrid search on relevance and repetition, giving up part of the plain rerank's gain for a more varied page.

Query understanding with categories

For queries like "kitchen faucet replacement", recognizing the product category can improve results. Following Doug Turnbull's rule, Qdrant filters only when very sure and otherwise boosts matching products. Choosing category names directly from products gave almost no gain. What helped was clustering products, letting a cheap LLM name each group, and using Jev to check the names.

At indexing time Jev tags each product with categories stored as payload. At search time it scores each category against the query: at 0.9 or more the search filters to that category, from 0.5 to 0.9 it boosts matching products, and below that it does nothing. The filter and boost run in Qdrant as a payload filter and a score formula on top of hybrid search.

On 300 Amazon-C4 searches, tiered filters at 0.9 with boosts from 0.5 worked best, and every setup beat plain hybrid search on average. Jev adds half a second to two seconds per query, depending on how many categories it reads. Small classifiers trained on Jev's product tags handled 37% of queries with boosts only, skipping Jev with slightly higher average nDCG@10.

Short queries are riskier

The Amazon-C4 queries are long and LLM-generated. Qdrant also tried a few short queries without relevance labels, so the authors say to read them as examples, not measurements. For "apple", Jev filtered on Food & Beverage at 0.91, and the page became all food, losing the MacBook and other results. For "kitchen faucet replacement" the Plumbing Parts filter (0.98) helped. For "running shoes that aren't nike" the athletic footwear category (0.98) was right but barely helped, since "not Nike" cannot be expressed as a category.

According to Qdrant, filtering works when the category is the whole intent, while short queries are more ambiguous and a filter hurts them most, because once it removes products MMR cannot bring them back. For teams building search, the results suggest using Jev's scores for ordering and boosting by default, and reserving hard filters for very confident cases.