Datadog adds monitoring for OpenAI Codex usage and impact
Datadog has introduced an OpenAI Codex integration that gives engineering leaders visibility into how developers use the coding agent, what code they accept and how they respond to its review feedback. The integration reports Codex activity as Datadog metrics and includes an OpenAI Codex Overview dashboard.
- Datadog reports per-user OpenAI Codex activity as Datadog metrics.
- The dashboard tracks threads, turns, credits, tokens, models and clients.
- The integration measures Codex-attributed lines of code added and removed.
- Code review metrics include P0, P1 and P2 comments and developer reactions.
- Codex data can be sent to AI Impact for AI-assisted and non-AI comparisons.
A broader view of Codex adoption
Datadog says organization-wide usage totals provide an incomplete picture of AI coding adoption. High activity could be concentrated among a small group of engineers, or developers could rely heavily on one Codex client while rarely using another.
The integration collects per-user Codex activity from the Codex Analytics API once per day. Its dashboard tracks threads, turns, consumed Codex credits and tokens, with filters for workspace, account, user, model and Codex client.
Threads represent Codex conversations started by an engineer, while turns represent messages sent across those threads. Datadog says the openai.codex.usage.threads and openai.codex.usage.turns metrics offer complementary measures of engagement, including activity trends and user rankings.
Tracking tokens, models and clients
The dashboard breaks token use into cached input, uncached input and output tokens. Datadog emphasizes that credits reflect Codex credit consumption rather than dollar amounts, and that the integration does not collect spend or currency data.
Model-level views show input and output token usage by model and speed tier, as well as rankings by total token consumption. Leaders can use those views to see which models account for the largest share of use and how patterns change over time.
Datadog also tracks threads and turns across Codex clients including the CLI, desktop app, GitHub, web and IDE integrations. Because the Codex Analytics API does not provide token data by client, those comparisons focus on threads and turns.
Measuring code contributions and review feedback
Usage counts do not show whether Codex output reaches the codebase. Datadog’s integration therefore collects lines of code added and removed that are attributable to Codex. Leaders can compare those figures with threads, turns and token consumption to understand how Codex fits into different development workflows.
The integration also tracks completed reviews and review comments by priority, including P0, P1 and P2. It records developer responses such as upvotes, downvotes, other reactions and replies, allowing teams to compare review volume with engagement over time.
Datadog cites its own use of Codex as an example. In a test using historical incidents, Codex found more than 10 cases, or roughly 22% of the incidents examined, where the engineers who owned the incidents said its feedback would have made a difference. Datadog says more than 1,000 of its engineers use Codex regularly.
Why the integration matters for engineering teams
The dashboard is designed to connect AI coding activity with software delivery outcomes. Teams can send Codex engagement data to Datadog AI Impact and compare software delivery performance between AI-assisted and non-AI work.
That gives engineering leaders more than a single adoption total. They can examine who uses Codex, which models and clients are involved, how much code is attributed to the tool and whether developers engage with its review comments. Datadog presents these signals as quantitative context for evaluating AI-assisted development alongside direct developer feedback.
Datadog: Monitor OpenAI Codex usage and impact with Datadog
