New Relic Brings AI Observability to OpenTelemetry Data Without Duplication

The problem New Relic says it addresses

New Relic argues that as engineering teams standardize on OpenTelemetry for generative AI architectures, a structural data disconnect creates visibility gaps. According to the company, this fragmentation can lead to unmanaged applications and unmonitored LLM token usage that drains cloud budgets. It can also leave OTel trace streams cut off from core safety pipelines.

Without unified tracking, New Relic says, production open-source AI services are exposed to undetected prompt injections, PII leakage and toxicity threats, which it describes as financial and compliance risks.

The company's answer is a "Normalize on Read" architecture. Instead of duplicating data or requiring developers to rewrite code, the platform aligns different schemas at query time.

Cost governance and model comparison

The feature provides account-wide Model Inventories that merge open-source and native APM agent telemetry into dashboards. Metrics named by New Relic include total token consumption, request latency and error rates.

A Model Comparison interface uses a per-column query resolution engine for what the company calls cross-instrumentation benchmarking. Engineers can compare side by side a legacy service instrumented with a proprietary APM agent and a newer microservice using open-source OpenTelemetry, with cost, latency and quality data on a single screen. The aim is to help teams pick the most cost-effective LLM configuration.

Normalize on Read and stateless processing

New Relic contrasts its approach with platforms that normalize data on write, converting or duplicating OTel spans into proprietary event formats at the ingestion gateway. In its view, that raises storage fees, processing overhead and cloud costs.

In New Relic's design, OTel telemetry is written natively to standard Span tables with no translation at ingestion. At runtime, the UI query engine detects the telemetry source and adapts its NRQL templates to query the Span table directly.

The company also points to the OpenTelemetry GenAI semantic conventions. A single OTel span carries both user prompts (gen_ai.input.messages) and LLM responses (gen_ai.output.messages) in one payload. By contrast, traditional vendor agents often emit prompts and completions as separate events that need stateful tracking and external caching to reassemble. A dedicated OtlpEvaluationHandler processes the spans statelessly, which New Relic says removes Redis caching and session reassembly dependencies and lowers total cost of ownership.

Finding shadow AI and extending safety checks

New Relic describes "Shadow AI" as microservices that adopt open-source LLM libraries without the knowledge of platform or security leads. To surface them, a TraceProcessor extension in the ingestion pipeline scans incoming OTel traces for gen_ai. semantic attributes. When an LLM interaction is detected, the pipeline extracts the entity GUID and publishes an aiEnabledApp:true tag to the global Entity Platform, with no manual tagging. The service then appears in the central AI Monitoring UI, giving platform teams and SREs an inventory with filtering, cost segmentation and baseline alerts.

On security, the feature routes open-source trace data into the core evaluation stream. The company says this enables automated PII masking, real-time prompt injection defense and toxicity screening on live traffic, without complex security middleware.

What it means for teams

New Relic frames the release around three roles. DevOps and platform teams get unified dashboards for aggregate LLM spend. Security and compliance officers get auto-tagging of unmapped apps and routing to ai_evaluation pipelines. OTel AI developers get analysis of prompt payloads and token volumes without changing their codebase.

According to the company, deployment requires no custom code rewriting or vendor-specific SDK lock-in, because the platform ingests standard OpenTelemetry GenAI semantic conventions. Teams can keep using their preferred open-source libraries. The source does not give pricing or availability details.