Orange Business builds Live Intelligence AI platform on LangChain and LangSmith

Why Orange Business did not build from scratch

Live Intelligence is Orange Business's flagship AI platform. Its Portal lets employees create simple AI agents and trigger basic API calls, which works for entry-level personal tasks. The company wanted to go further: agents that orchestrate complete business processes across fragmented enterprise systems, govern who can access what, and run reliably in production at scale.

According to the case study, reproducing the orchestration, deployment, monitoring and observability infrastructure for such agents would have required over a year of development. Orange Business looked for an existing foundation, and Live Intelligence Studio came out of that search.

Orange Innovation, the Orange Group's innovation scouting arm, evaluated roughly 20 major cloud providers and specialized AI vendor solutions. It chose LangChain for its model neutrality and portability. That allows the team to keep full ownership of the agent architecture and manage workloads in line with European data sovereignty requirements. LangSmith was the strongest fit for evaluations and observability.

"The observability capabilities in LangSmith were significantly ahead of what we could realistically build internally," said Mathieu Ducrot, Head of AI Products at Orange Business. "Building it ourselves would likely have delayed our roadmap twelve to eighteen months."

Studio, Portal and Console

Live Intelligence has three components, each aimed at a different user.

• Live Intelligence Studio is for AI engineers and agent developers who design complex, cross-system workflows. It includes an LLM-as-a-Service catalog of models hosted on Orange Business infrastructure and public cloud, an agent builder powered by LangSmith Fleet, MCP support, agent observability through LangSmith and orchestration tooling.
• Live Intelligence Portal is the interface for business end users, such as HR teams drafting job descriptions, legal teams reviewing NDA clauses, software teams generating test scripts and helpdesks building IT support agents. Workflows built in Studio can be deployed directly into Portal.
• Live Intelligence Console gives enterprise administrators usage monitoring, performance tracking and enforcement of security and compliance policies.

On top of this, two registries handle governance across the organization. The Agent Registry lets teams discover, reuse, version and govern agents and prompts, so that agents become shared assets with lifecycle management. For prompts, Orange uses LangSmith Context Hub to keep different versions per environment, which allows prompt changes to be tested and validated before reaching production and without changing code.

The MCP Registry is a centralized catalog of trusted tools, connectors and external capabilities. Agents reach enterprise systems such as Microsoft 365, CRMs, internal APIs and custom services through a controlled, auditable layer. According to the case study, this moves business logic into the platform layer and out of prompts, so each team does not have to solve authentication, security and governance again.

A three-layer evaluation setup

Orange Business runs a 3-layer evaluation in production, built on LangSmith.

• RAG quality: golden datasets with question-answer pairs, expected source documents, retrieval recall, grounding quality and answer faithfulness.
• Agent behavior: tool selection accuracy, task completion rates, refusal behavior, robustness against prompt injection and adherence to business constraints.
• Production trace monitoring: latency, cost, model usage, failed runs, human feedback and escalation cases.

Ducrot said debugging used to be difficult because failures were hidden inside model behavior. "With LangSmith, engineers can inspect each step of the execution and understand whether the issue comes from retrieval, the prompt, the model, the tool call, or the business logic," he said. AI engineers, backend engineers, product managers and QA can look at the same traces and evaluation results.

Orange also runs regression evaluations before pushing changes to production. Engineers can compare agent versions, replay historical traces and validate behavior before deployment. For workflows with several agents and tool calls, the full execution tree helps them find where a workflow failed, which agent or tool was running, what input and response were involved, and how long individual steps took. Orange also builds datasets of trace examples showing good or bad agent behavior for offline evals and continuous prompt refinement.

Results and next steps

The case study says the system has improved operational efficiency across teams and given engineers and subject matter experts a shared workflow for agent development. "For us, the technical team owns the application and agentic workflow implementation, while the business team can actively participate in prompt improvement and quality evaluation," Ducrot said, adding that LangSmith turns prompt iteration, tracing and evaluation into one continuous improvement workflow.

Orange plans to expand team collaboration into online evals for richer feedback cycles. The team is also working on automating more complex business processes, for example agents that support the entire insurance claim process with CRM and legal workflow integrations, and an agent that monitors security sensors in a port for field-engineer emergency intervention.

The account comes from LangChain, which sells the tools involved, so the figures and benefits are the vendors' and Orange's own claims. It does show how a large enterprise approaches agent governance: shared registries, versioned prompts and evaluation gates before deployment.