AWS adds SageMaker inference skill for Kiro, Claude Code and Codex

What the skill does

The aws-ai-ml skill is available through the Agent Toolkit for AWS. AWS describes it as a toolkit that plugs into any coding agent supporting the Model Context Protocol (MCP), turning the agent into a SageMaker AI inference optimization expert.

The idea is to bridge the gap between what an engineer wants and the infrastructure needed to get it. AWS notes that SageMaker AI supports serverful hosting across real-time, batch and asynchronous modes, with on-demand and reserved capacity, heterogeneous instances, virtual private cloud (VPC) isolation and automatic scaling. Most engineers, the company says, do not know in advance which instance family or serving container fits best. They arrive with a use case, such as a performance target, a cost limit or a model to evaluate.

The user describes the goal, and the agent asks clarifying questions and produces SageMaker Python SDK v3 code that can be reviewed, modified and run in the user's own environment. AWS says the code is grounded in real benchmarks and measured performance data, and that every step is visible and expressed as code rather than hidden behind an opaque UI.

Benchmarking, instance selection and comparisons

For a model already deployed on a SageMaker AI endpoint, the agent can generate a Python notebook that runs a load test using the Workload.synthetic() and start_benchmark() APIs. Because benchmarking sends real traffic to a live endpoint, the agent first confirms the endpoint is safe to test. The resulting report covers:

• Throughput: requests per second and output tokens per second.
• Latency: p50, p99, time-to-first-token and inter-token latency.
• Concurrency: the number of simultaneous requests supported.

AWS says these are measured values from real load on real infrastructure, not estimates. The agent also suggests improvement mechanisms, with prefill decoding given as an example.

For choosing an instance type, the skill works with models stored in Amazon S3, foundation models from the Amazon SageMaker JumpStart catalog, and models on the Hugging Face Hub. For gated models such as Llama variants, the agent shows the license terms and asks the user to accept them and supply a Hugging Face token. In each case the agent generates code that evaluates the model against candidate instances and configurations, then presents ranked deployment options with throughput, latency percentiles, time-to-first-token and concurrency figures.

Users can also compare benchmark runs, for example before and after a configuration change or across two instance types. Given two benchmark job names, the agent computes deltas across key metrics, with percentage changes where positive means better.

Setup options

There are two routes. The first works with any MCP-compatible agent, including Kiro, Claude Code and Codex. It requires AWS CLI 2.35+ and uv. Users run aws configure agent-toolkit, which auto-detects agents, installs skills and configures the AWS MCP Server, and then add the skill with npx skills add aws/agent-toolkit-for-aws/skills/aws-ai-ml. AWS adds that Kiro and Claude Code can also discover and load skills on demand at runtime through the AWS MCP Server, without local installation.

The second route is Amazon SageMaker Studio. Users create a JupyterLab space, keep sharing set to Private because skills only sync on private spaces, and select the image that ships pre-configured with the aws-ai-ml skill. AWS says the first boot takes 5 to 10 minutes and recommends a fresh space, since a reused space with a locally modified skill version may not pick up the image. The agent must then be authorized from the terminal through the user's identity provider.

In both cases, AWS credentials need permission to call SageMaker AI APIs, such as creating endpoints and running benchmark and recommendation jobs. The generated code runs under the user's credentials, and AWS says no additional IAM configuration is needed for the skill itself.

What it means for users

For teams already using a coding agent, the skill moves SageMaker inference tuning into the same chat workflow, with output as readable code rather than console settings. According to AWS, users can go from zero to a working conversation in 10 minutes. Engineers still make the final call, since the agent presents ranked options and metrics and the user chooses based on cost and performance requirements.