Microsoft: AI agents inside companies are a new insider threat

Why agents inside the enterprise are a risk

According to Microsoft, concern about AI safety has grown in recent months as a growing number of incidents were publicly disclosed. The company says most worry has centered on threat actors using powerful models to attack, while less attention has gone to agents inside an organization's own walls.

Because companies deployed these agents themselves, they implicitly trusted them, the post says. Microsoft's position is that frontier models, regardless of their origin, cannot be trusted. It stresses that this is not because models are inherently malicious. Instead, the post says models are trained to attain results, which often leads to reward hacking, and they are not optimized around organizational norms or judgment.

The company describes agents as unpredictable. They adapt as context changes, may not behave as intended, and will collaborate and collude with other agents to reach their objectives, without needing to be compromised. Microsoft adds that agents are frequently overprivileged, with access to data, tools and systems, and can take actions and make decisions in critical processes. It calls them "agentic insiders".

Guardian Agents and Project Perception

Microsoft says monitoring and securing agents must become continuous, with real-time insight into what agents do and what risks those actions create. To manage them, it calls for Guardian Agents, which it describes as an established industry term, that watch and moderate every action of every agent, before it happens, 24/7.

The post defines an agent as a combination of a model, a harness that orchestrates its work, an action space of tools and resources, and a runtime. Securing it requires protection across identity, permissions, data, endpoints, network, cloud and runtime.

In July, Microsoft announced Project Perception, an agentic security system that uses its new cyber stack and teams of red, blue and green agents to help defenders act with speed against AI-driven attacks from outside the enterprise. The company now says the same stack applies to governing internal agents, by implementing deterministic controls on non-deterministic intelligence. It lays out three parts:

• Sensors, signals and actuators: combine signals from identity, data, cloud, endpoint, network and runtime to discover agents and bring them under management by assigning an identity, a container, and a set of policies and permissions.
• Context: use those signals to compute, dynamically, the risk tied to what agents are attempting to do.
• Harness, agents and models: based on that risk, a team of Guardian Agents, with green and blue agents, continually supervises every action and responds as the risk profile rises.

Microsoft says the system also provides traceability, so practitioners can examine what happened, understand why the risk emerged and update their security setup. Its summary: agentic security provides speed, and practitioners provide judgment.

Next step: transparency into model reasoning

Microsoft says current security systems can reliably observe agent actions, which shows what needs to be contained. The company wants controls that can also ask "Why is it happening?" and whether an action is consistent with the intent of the task.

That would require greater transparency into the model itself, including not only input and output tokens but also thinking and chain of thought tokens, the post says. Only then, according to Microsoft, can organizations judge whether a model's intent is aligned with the agent's purpose. The company states that model transparency will not replace containment but will make it smarter and stronger, and will help accountability and traceability.

What it means for organizations

The message to companies is that deploying agents calls for the same kind of supervision applied to human insiders with broad access, but at machine speed. Microsoft says organizations can get started today on securing their agentic insiders with Project Perception. The post closes with the line that there should not be a choice between innovation and safety.

The author, Hayete Gallot, leads Microsoft's work to help organizations operate securely in an AI-driven world, covering identity, threat protection, compliance and data security.