Datadog adds Prompt Management to Agent Observability for LLM apps

Datadog has introduced Prompt Management in its Agent Observability product, a centralized registry for creating, versioning, deploying and retrieving LLM prompts independently of an application's release cycle. Combined with Prompt Tracking, it ties each prompt version to the LLM calls that used it.
- Datadog added Prompt Management to Agent Observability, a registry to create, version, deploy and retrieve LLM prompts.
- Applications fetch prompts at runtime via a stable Prompt ID, so new versions can ship without a new application build.
- Versions can be deployed per environment, with targeting rules built on Datadog Feature Flags.
- Combined with Prompt Tracking, versions can be compared by call volume, latency, token usage and cost.
- Runtime retrieval supports Python; Agent Observability is free up to 40,000 LLM spans per month.
A single registry for prompt versions
According to Datadog, prompts are a key part of LLM application behavior, yet teams often keep them in application code, notebooks or separate configuration files. That can make prompt changes depend on application releases and make it hard to tell which version runs in each environment.
In Prompt Management, each managed prompt has a stable Prompt ID and a version history. Applications keep referencing the same identifier while the prompt is updated. When a prompt is edited, Datadog preserves earlier versions instead of overwriting them, and version history and comparisons let teams review changes and return to an earlier version if needed.
The Prompts page brings managed prompts and prompts captured through Prompt Tracking into the same catalog. If Agent Observability already tracks a prompt in an application, it can be registered for management rather than recreated. Teams can also create prompts directly in Datadog and define variables for content the application supplies at runtime.
Deploying prompts without shipping a new build
Prompt Management separates prompt deployment from application deployment. Instead of embedding each revision in code, the application retrieves a managed prompt at runtime using its Prompt ID, and Datadog resolves the request to the appropriate deployed version. A team can publish a new version and deploy it to an environment without a new application build. If a change produces unwanted results, an earlier version can be deployed without rolling back unrelated application changes.
Applications can also keep a local prompt as a fallback in case managed prompt retrieval fails. Datadog says runtime retrieval is supported in Python, and points to its documentation for prerequisites, supported ddtrace versions, authentication and fallback behavior.
Versions of the same Prompt ID can be deployed to different environments, for example one version in development, another in staging and a stable one in production. Applications retrieve the version tied to their environment, while developers can request an exact version to reproduce or validate behavior.
For finer control, Prompt Management builds on Datadog Feature Flags to support targeting rules. Within an environment, one version can stay the default while another is served to requests matching attributes the team defines.
Linking prompt changes to performance
When Agent Observability is enabled, Prompt Management works with Prompt Tracking to associate managed prompt versions with LLM spans. From the Prompts page, teams can compare versions by call volume, latency, token usage and cost, review differences between prompts, and jump to traces that used a particular version.
Datadog gives the example of latency or token usage shifting after a prompt update: a team can identify the version tied to the affected calls and inspect the matching traces. Prompt Tracking also allows filtering Trace Explorer by prompt name, ID or version.
The two capabilities remain separate. Applications can retrieve managed prompts without enabling Agent Observability, while Agent Observability supplies the LLM spans and prompt metadata needed to analyze usage and performance.
Availability and what it means for teams
Datadog says Agent Observability is free for Datadog customers submitting up to 40,000 LLM spans per month. Those without an account can sign up for a 14-day free trial.
For teams running LLM applications, the change offers a way to iterate on prompts on their own schedule while keeping a record of which version served which traffic. According to Datadog, the aim is to help teams investigate how prompt changes relate to application behavior.
Datadog: Manage and deploy LLM prompts with Datadog


