Atos tunes small models with Red Hat AI to rival GPT-4.1 on German text

A task that more context cannot solve

The Atos project focuses on parsing German-language social media content across 14 categories, such as indicators of harmful content. Downstream systems depend on the output, so the model has to return its insights in a consistent, defined JSON structure. The solution also had to meet strict sovereignty requirements, with control over the data, the models and the training pipeline.

Red Hat notes that retrieval-augmented generation (RAG) has become an important way to connect AI models to proprietary information, but not every problem is solved by giving a model more context. Here the challenge was to get a model to interpret primarily German-language content, extract the right information and reliably return it in the required structure. According to Red Hat, that means changing how the model behaves, not just what information it can access.

With Red Hat AI, Atos used synthetic data generation to build the high-quality datasets needed to fine-tune small models for structured extraction. Red Hat says this addresses the scarcity of domain-specific data, a common hurdle in fine-tuning.

Results: small models against GPT-4.1

The use case was designed with potentially millions of records per day in mind, which makes inference cost and latency important. The key question was how small a model could be and still do the job.

• Fine-tuning improved Phi-4 Mini's overall evaluation score by as much as 26.4% compared with its base-model baselines.
• The tuned model reached approximately 99% valid JSON through fine-tuning alone, without an additional harness to enforce the output structure.
• The fine-tuned Qwen 2.5 7B model outperformed GPT-4.1 in the evaluation.
• The fine-tuned Phi-4 Mini came very close to GPT-4.1 despite its much smaller size.
• Phi-4 Mini also outperformed NuExtract 2.0 8B, a model purpose-built for structured extraction and more than twice its size, by up to 22.6% in overall score.

Red Hat stresses the limits of the finding. It does not mean smaller models will beat frontier models on every workload, only that the largest model is not necessarily the right choice for every task.

From data sovereignty to AI sovereignty

Red Hat argues that sovereignty is not only about keeping data inside a geographical border. Enterprises increasingly need control over the models themselves: where they run, how they are adapted and how they fit into business processes. An open model, it says, can be customized for a company's own workloads and deployed on infrastructure the company controls.

This matters more with agentic AI, according to the company. Enterprise agents often orchestrate many specialized tasks, such as extracting information, classifying requests, producing structured outputs and interacting with enterprise systems. These tasks often call for consistent, task-specific behavior rather than the broad capabilities of a frontier model. Instead of sending every task to a large frontier model, companies can pick the right model for each workload and specialize smaller ones where it makes sense.

What it means for enterprises

For high-volume production, Red Hat says fine-tuning a small model on a well-defined workload delivers the right precision without the overhead of a general-purpose frontier model, and that this dramatically improves the economics of running AI at scale. It positions fine-tuning as a complement to RAG: RAG supplies relevant enterprise knowledge, while fine-tuning teaches a model more specialized and consistent behavior.

The results come from one German-language extraction task and are reported by Red Hat, not by an independent evaluation. Within that scope, Atos concludes that a small, tuned model can match or exceed frontier model performance on specific enterprise tasks while providing the control that sovereign AI requires. Red Hat also points readers to its technical blog post on how Atos approached structured outputs, as well as Red Hat OpenShift AI and AI quickstarts.