Elastic moves Jina models to NVIDIA RTX Pro 6000, reports up to 8x faster embedding

Migration from L40 to RTX Pro 6000

Elastic offers Jina AI models, which cover reader, OCR, embedding and ranking tasks, through two managed services. The Jina API handles direct calls to Jina models, while Elastic Inference Service (EIS) provides pre-configured, GPU-accelerated endpoints natively integrated with Elasticsearch.

To keep up with growing agentic AI workloads, Elastic is moving both services from NVIDIA L40 GPUs to NVIDIA RTX Pro 6000 Blackwell Server Edition GPUs. Some Jina API models already run on the new hardware, and according to Elastic, customers using them get the speed gains without changing anything. The company says it is continuing to migrate more models of the Jina API and EIS, so more customers should see improvements as each hosted model moves.

The models behind the services

Elastic describes the Jina models as small, frontier-grade models that preserve meaning from ingestion to retrieval. The post highlights several claims about them:

• OCR: jina-ocr-v1 parses text and scripts in more than 100 languages and, according to Elastic, is the leading under-600M active parameter model on the olmOCR-bench benchmark.
• Embeddings: Jina omni embedding models handle text, image, audio and video in more than 100 languages. Elastic says jina-embeddings-v5-omni-small is the best performing openweight omni model under 2B parameters on MTEB.
• Reranking: Jina-reranker-v3.5, a 600M parameter model, beats the 4B parameter Qwen3-Reranker-4B model on the STaRK structured data benchmark, according to Elastic.

Why mixed workloads mattered

Elastic argues that small models do not by themselves simplify capacity planning. Jina API and EIS are shared services, so large jobs run next to small ones on the same GPUs. A single reranking request can carry a long list of candidate documents, while a query behind it may need one quick answer. If the large job occupies the GPU, the small request can wait in a queue and its latency rises.

When benchmarking the RTX Pro 6000, the company says it cared most about whether small requests could stay fast while large jobs ran beside them.

What Elastic measured

Elastic compared production traffic before and after the move and ran controlled tests for cases production cannot isolate. It focuses on latency percentiles, noting that at the 99th percentile, 99 of every 100 requests finished faster than the value shown. The figures below compare the previous GPU setup with the RTX Pro 6000.

• Embedding (production): 99th-percentile latency fell from 5.86 s to 0.73 s, about 8x.
• Reranking (production, with about 40% more average request volume): 90th-percentile latency fell from 732 ms to 171 ms, 4.3x.
• Large reranking job (controlled test, same software): completion time fell from 702 ms on three L4s to 160 ms on one GPU, 4.4x.
• Small reranking requests alongside large jobs (mixed-traffic test on a shared GPU pool): latency fell from 708 ms to 66 ms, 10.7x.
• Document OCR with jina-ocr-v1 (same set of real document pages): time to first output token fell from 0.50 s to 0.20 s, 2.5x, and page conversion speed rose from 454 pages/hr to 1,671 pages/hr, 3.7x faster.

What it means for customers

Elastic says that on the RTX Pro 6000, small requests stayed fast while large jobs ran beside them. For customers, it states, that means a query does not slow down because a big job is running on the same service. Because the migration is ongoing, the gains apply to models already moved, and Elastic expects more customers to benefit as additional hosted models follow. The company also notes that the release and timing of any described features remain at its sole discretion.