Arm maps the software stack for agentic AI on its new AGI CPU

Why the software around the model matters

According to Arm, generative AI put models at the center of the infrastructure conversation, but agentic AI shifts that balance. When agents retrieve data, invoke tools, execute code and coordinate multi-step workflows, the surrounding software becomes just as important as inference itself.

Arm says it built Arm AGI CPU for these demands. It describes the chip as a new class of Arm Neoverse based processors, designed for highly concurrent agentic workloads such as orchestration, tool execution, data retrieval and application services. At OCP Global Summit, Arm introduced a new CPU evaluation methodology with support from Fujitsu, Meta and OpenAI. According to Arm, independently verified results show the chip delivers up to 2x performance per rack versus a leading x86 server processor.

The company argues that CPU innovation alone is not enough. It points to a mature Arm64 software ecosystem, with broad native support across cloud-native and AI software and Neoverse based CPUs already deployed at scale on leading cloud platforms.

The stack and the different needs of agents

Arm describes the agentic AI software stack as covering agent frameworks and orchestration, tool execution, memory and retrieval, model serving, and cloud-native infrastructure. Technologies named in the post include LangGraph, LangChain, Firecracker, Redis, Elasticsearch and vLLM. Developers can check software availability and supported versions through the Arm Software Ecosystem Dashboard, which has a dedicated Agentic AI category.

The post stresses that agents differ in what they demand from infrastructure:

• Coding agents run code, compile software and run tests, which puts more load on CPU performance and sandboxing.
• Enterprise workflow agents depend on API interactions, database transactions and orchestration.
• Research agents rely on search, retrieval and memory management.
• Personal assistants prioritize interactive responsiveness.
• Robotics applications add needs for simulation and latency-sensitive coordination.

Arm concludes that infrastructure should be evaluated across the complete workflow, not through model inference performance alone.

Performance figures and the customer service demo

Arm cites several results for specific parts of the stack. Independent testing of Arm AGI CPU by Signal65 showed up to 45% lower sandbox creation latency compared with AMD EPYC 9965, according to the post. For vector search, Redis 8.10.1 benchmarks showed AWS Graviton5 delivering 16 to 27% more context fetches per dollar than x86. The advantage was 16% on a 570K-document dataset under normal load and 27% on a 3.3M-document dataset under heavy load.

Arm also points to recent Elasticsearch benchmarks on Arm-based infrastructure and to its vLLM developer resources, which cover deployment, quantized inference and performance tuning. OpenInfer is working with Arm to demonstrate its Loom platform running AI models across multiple Arm-based CPUs.

To show the pieces working together, Arm built a multi-agent customer-service application on the CPU using LangGraph and LangChain. A CPU-hosted small language model classifies incoming requests and routes them to agents for meeting scheduling, complaints, product inquiries and refunds. Those agents combine CPU-based inference, enterprise data retrieval, external model APIs and tool execution. Tasks include scheduling meetings, generating personalized product brochures and starting refund processing through enterprise systems.

Resources for developers

Arm Learning Paths offer tutorials on building AI agents with llama.cpp, running AI-generated code in isolated Firecracker microVMs, implementing RAG applications with LangChain and FAISS, and deploying vLLM model serving on Arm-based platforms.

The post also highlights AI agent skills. AWS has introduced Graviton developer skills, while Arm is contributing Arm Performix, Arm Performix CI and Arm Migrate Ease through OpenAnolis SkillHub. Arm says these bring platform-specific knowledge and optimization guidance into AI-assisted development workflows.

What it means for developers

For teams building agentic applications, the post positions Arm64 as a platform where familiar open-source tools already run, with the Software Ecosystem Dashboard as a starting point. Arm also advises developers to find where an agent spends its execution time, for example in sandboxes, databases and APIs, or retrieval and caching, before tuning the stack. Software vendors working on agent frameworks, tool execution, data infrastructure or model serving are invited to contact Arm's software ecosystem team. The performance figures come from Arm and its cited testers, so they reflect the specific setups described in the post.