Artificial intelligence news.
Straight from the original announcements.

LangSmith adds alerts for errors, latency and LLM quality issues

November 18, 2026, 20:33By the AI News Bites Editorial TeamSource: LangChain
LangSmith adds alerts for errors, latency and LLM quality issues
Image: LangChain

LangChain has launched Alerts in LangSmith, giving developers a way to monitor LLM applications and agents for production problems in real time. The feature can flag errors, latency changes and drops in feedback scores.

Key points
  • LangSmith Alerts monitor error count and rate, average latency and average feedback score.
  • Developers can filter alerts by model, tool call or run type.
  • Alerts support 5 or 15 minute aggregation windows and configurable thresholds.
  • Notifications can be sent through PagerDuty or a custom webhook.
  • LangChain plans alerts for run count, token usage, relative changes and custom time windows.

Alerts target reliability and output quality

LangChain says LangSmith Alerts are designed to help developers catch failures before they affect users. The feature is available for applications that send production traces to LangSmith, while developers new to tracing can begin with tracing in LangSmith.

Alerts can be based on three groups of metrics: error count and rate, average latency, and average feedback score. These metrics address both operational failures and changes in the quality of an LLM application's output.

Filters and time windows narrow the signal

Developers can apply filters to focus alerts on particular subsets of runs. LangChain gives model, tool call and run type as examples of filtering options.

Each alert can use an aggregation window of 5 or 15 minutes, together with a threshold that controls its sensitivity. This allows teams to tune alerts around the parts of an application they want to monitor, rather than treating every run as the same.

LangChain also supports sending alerts to PagerDuty or connecting them to a custom webhook. The company gives sending notifications directly to a Slack channel as an example of how a webhook could be used.

Why monitoring is difficult for LLM applications

According to LangChain, agentic applications depend on external services including model providers, APIs, web search services and databases. Outages, rate limits and increased latency affecting those dependencies can reduce the user experience, making early detection important.

Quality problems create another challenge. Changes to prompts, models or inputs can produce unexpected results, and prompts that work well in controlled evaluations can regress in real-world use because user interactions differ. LangChain says feedback scores from user input or online evaluations can provide an early warning of these quality drops.

More alert types are planned

LangChain says it plans to add alerts for run count and LLM token usage. It also plans change alerts, which would allow developers to set a relative value, such as an alert when latency spikes 25%, as well as alerts covering custom time windows.

For users of LangSmith, the launch adds a monitoring layer that can connect application metrics with existing incident workflows. The planned additions suggest the company intends to expand beyond fixed thresholds and the current error, latency and feedback measures.

Related news

Mastra adds observability alerts for costs, errors and eval scores
Products and toolsNovember 19, 2026

Mastra adds observability alerts for costs, errors and eval scores

Mastra Platform now lets users create observability alerts for model costs, error rates, latency, token usage and evaluation scores. Notifications can be sent to Slack…

LangChain adds cross-agent tracking to help control coding costs
For developersNovember 18, 2026, 20:33

LangChain adds cross-agent tracking to help control coding costs

LangChain is expanding LangSmith to give teams a shared view of spending and activity across multiple coding agents. The company says the tools can help organizations…

LangSmith links coding agent spending to pull request outcomes
Products and toolsNovember 18, 2026, 16:00

LangSmith links coding agent spending to pull request outcomes

LangChain has added coding agent reporting to LangSmith, linking spending to GitHub pull requests and GitLab merge requests. The company says the feature is designed to…