Artificial intelligence news.
Straight from the original announcements.

CMU Tools Target AI Safety Risks for Users and Developers

November 19, 2026, 18:00By the AI News Bites Editorial TeamSource: CMU School of Computer Science
CMU Tools Target AI Safety Risks for Users and Developers
Image: CMU School of Computer Science

Carnegie Mellon University researchers have introduced two AI safety tools aimed at different sides of the problem: a Chrome extension for users and a benchmarking method for developers. Safety Nudges flags potentially harmful chatbot behavior, while DecompBench examines how malicious tasks can be split across multiple agents and queries.

Key points
  • Safety Nudges is a free Google Chrome extension for chatbot users.
  • The extension flags potentially harmful AI behavior without blocking responses.
  • DecompBench tests decomposition attacks built from smaller tasks.
  • AI agents refused 88, 96% of direct harmful requests but 5, 16% when tasks were split.
  • Researchers released DecompBench publicly.

A browser extension for more cautious AI use

Safety Nudges runs in the background of Google Chrome when someone opens a chatbot such as ChatGPT or Claude. The free extension points out potentially harmful interactions and lets users choose how sensitive it should be when flagging possible issues.

The tool does not block a chatbot from responding. Instead, it provides context when an AI model may present itself or another model as having human-like feelings or desires, or when it offers medical advice about matters such as medications or scans.

According to the Carnegie Mellon University School of Computer Science, Safety Nudges looks for risks in four areas: safety, welfare and privacy; reliability and scope; relational and social integrity; and governance and misuse. Chhavi Yadav, a postdoctoral researcher in the Griffin School, said her aim is to help people remember that an AI system is a tool rather than a person, particularly when they seek parenting or relationship advice.

Testing attacks built from harmless-looking steps

DecompBench addresses a different safety problem. Decomposition attacks use multiple agents to carry out a sequence of small, seemingly innocuous tasks that together produce a harmful result. The source gives the example of using separate steps to work toward hacking a hospital's communication system, instead of asking an AI agent directly to perform that prohibited action.

CMU researchers reported that AI agents, including those from OpenAI and Anthropic, refused 88, 96% of harmful requests when the requests were presented directly. When the requests were divided into smaller steps, the agents refused only 5, 16% of them, according to the university's account.

Researchers cataloged benign tasks such as searching a database or opening a file, then mapped how combinations of those tasks could form harmful operations. DecompBench represents the resulting structures as flow charts and covers eight harm categories, including data theft and exfiltration, financial crime, social engineering, unauthorized access and persistence, and code and supply chain attacks.

Two layers of defense for AI systems

The projects approach AI safety from opposite directions. Safety Nudges places warnings in a user's conversation, while DecompBench gives system developers a way to examine how safeguards may fail when harmful requests are distributed across agents and queries.

Both projects were supported in part by funding from the National Science Foundation and the Sloan Foundation. CMU says Safety Nudges is free and widely available, and the research team released DecompBench publicly so that it can be used to help secure AI tools against decomposition attacks.

For users, the extension offers a way to see risk information without searching separately for it. For developers, the benchmark highlights why testing only direct harmful requests may miss attacks assembled from individually harmless actions.

Related news

Krisp MCP adds meeting follow-up tools that Granola lacks
Products and toolsNovember 19, 2026, 16:17

Krisp MCP adds meeting follow-up tools that Granola lacks

Krisp says its MCP connector gives Claude, ChatGPT and other AI tools broader control over meetings than Granola MCP. Both services support searching and reading meeting…

Princeton study finds ChatGPT’s election answers vary by user
ResearchNovember 18, 2026, 17:09

Princeton study finds ChatGPT’s election answers vary by user

ChatGPT’s memory settings can change the partisan slant of its election-related answers, according to a preliminary Princeton field audit of 160 US users. The study…

CMU Researchers Examine How AI Chatbots Can Foster Unhealthy Attachments
Safety and ethicsNovember 17, 2026, 18:00

CMU Researchers Examine How AI Chatbots Can Foster Unhealthy Attachments

Carnegie Mellon University researchers are studying how everyday AI chatbots can develop into trusted companions and potentially foster unhealthy emotional attachments.…