Aleph Alpha says its Kolibri model beats larger rivals on agentic RAG tasks

Specialising a model for data it never sees

Kolibri was released last week as an open-weight LLM built for what Aleph Alpha calls mission critical sovereign applications. In a research post by Duncan Blythe, the company explains how it specialised the model for retrieval augmented generation (RAG), a central use case among its customers.

Aleph Alpha defines sovereign use cases as those in which control over model hosting, model values and alignment, training data distributions and compliance plays a central role. The customer must keep control over its own data and IP. According to the company, all its customer use cases qualify, and they generally come from the public sector and regulated industries such as automotive and technology.

From an analysis of its customers' portfolio, Aleph Alpha derived several requirements: models that understand and respond in English and German, strong performance across a broad set of skills, strong contextual performance in RAG tasks over out-of-distribution data, and proficiency with highly technical document types such as legal and technical documentation.

The company says it does not ingest any customer data point into its training pipelines without consent, and that Kolibri was not trained on any customer data. Customer corpora, databases and traces may contain sensitive information that should stay out of the training distribution.

Benchmark claims

Aleph Alpha reports that across seven agentic RAG benchmarks, five of them modelled on real customer deployments, Kolibri has the best average score of the leading open-weight models it compared. The company says it beats models of its own size as well as models that use several times more compute per token.

These results come from the company's own comparison, and the post as provided does not list the individual scores or the models used as reference.

Why the agentic harness matters

The post describes the standard 2026 approach to RAG: the LLM is connected to tools that give it access to external data, such as semantic search over internal corpora and document access, and is instructed in the prompt how to use them. The choice of tools, how the model may interact with them and how it copes with context-length overflow and complex tasks is called the agentic harness.

Key harness decisions listed by Aleph Alpha include:

• Search tools: search by meaning, keyword search, or hybrid search.
• Corpora and ETL: which documents are searched and at what granularity.
• Result set and surface: how many results the model sees and in what form, from titles only to full content.
• Supplementary tools: whether the model can open pages or full documents.
• Multi-linguality: how prompt, document and user languages correspond.

The decisions must fit the use case. In general knowledge applications, search by meaning makes sense. In technical documentation, keyword search is necessary because questions may reference identifiers such as "TC3x-235c", which the post says do not work well with search by meaning.

An engineer can combine both in hybrid search or expose several tools and let the model decide at runtime. A fixed choice reduces flexibility but also the variance and scope of errors. The post adds that a small model, chosen because of limited GPU compute at runtime, may lack the reasoning depth to handle a flexible harness.

A worked example

The post includes a trace for a technical documentation harness with a semantic search tool and a keyword search tool (BM25 with an identifier-preserving tokenizer), a document corpus and a structured errata table. The user asks for the maximum clock frequency of the TC3x-235c and any known errata.

The model first runs a semantic search, notices that the embedding treats TC3x-235c, TC3x-237a and TC3x-233b as near-identical and returns the wrong parts, then switches to an exact keyword match and also queries the errata table. The keyword search returns the TC3x-235c datasheet entry and an erratum noting that PLL lock time exceeds spec at fSYS > 280 MHz. According to the post, this kind of reasoning is not available in earlier or smaller models.

What it means for customers

For public-sector and regulated customers, the pitch is a model tuned for RAG over their own documents in English and German, without their data entering training. How well that holds up outside Aleph Alpha's benchmarks will depend on the detailed results and on independent testing.