Artificial intelligence news.
Straight from the original announcements.

AWS adds inference recommendations to SageMaker Studio, Python SDK and coding agents

November 20, 2026, 01:34By the AI News Bites Editorial TeamSource: AWS
AWS adds inference recommendations to SageMaker Studio, Python SDK and coding agents
Image: AWS

Amazon Web Services has extended its SageMaker AI inference recommendations beyond the original API, making them available in SageMaker AI Studio, the SageMaker Python SDK v3 and coding agents. According to AWS, the tool can now also size LoRA adapter deployments and find the highest concurrency an endpoint can sustain under a latency target.

Key points
  • Inference recommendations are now available in SageMaker AI Studio (July 2026) and the Python SDK v3, version 3.17.0 and later (August 2026).
  • September 2026 updates add LoRA adapter support and an SLA-based concurrency search using the max-concurrency-under-sla recipe.
  • The aws-ai-ml agent skill, available through the Agent Toolkit for AWS, brings SageMaker inference expertise to agents such as Kiro.
  • According to AWS, setup of the skill takes about 10 minutes and it works with any MCP-capable agent.

Recommendations where engineers already work

AWS describes the changes in part 2 of its review of 2026 SageMaker Inference launches. The post follows one hypothetical team that launched a customer support assistant on a 70B-parameter model behind a single SageMaker AI endpoint, and by the fall needs more product lines, a larger reasoning model, a second Region and a flat GPU bill.

Inference recommendations first launched in April as an API, automating the choice of instance, container and optimization. In July 2026, AWS added a guided, low-code workflow in SageMaker AI Studio. Users choose a use-case profile (Interact, Generate, Summarize, or Custom), an optimization goal (minimize latency, maximize throughput, or minimize cost) and a model from SageMaker JumpStart, Amazon S3, the Model Registry or an existing SageMaker model.

According to AWS, SageMaker AI then benchmarks candidate configurations on real GPU infrastructure using NVIDIA AIPerf. It applies techniques that match the goal, such as speculative decoding for throughput or kernel tuning for latency. Results are ranked by TTFT, inter-token latency, throughput and cost, and the chosen configuration can be deployed to a real-time endpoint from Studio.

In August 2026, the feature arrived in the SageMaker Python SDK v3, in version 3.17.0 and later. Teams can benchmark a live endpoint against their own traffic, generate ranked recommendations and deploy the top configuration in a single call. Recommendation jobs can run in parallel to compare serving frameworks such as LMI and vLLM, and a completed job can be resumed so deployment can happen in a different notebook or pipeline stage without re-running benchmarks.

LoRA adapters and SLA-based concurrency search

In September 2026, inference recommendations gained support for LoRA adapters. AWS notes that each adapter kept in GPU memory reduces the number of concurrent requests an endpoint can serve, and that the right balance depends on instance type, adapter count and traffic profile. Adapters can be supplied as Model Package ARNs, for adapters produced by SageMaker fine-tuning, or as S3 URIs, for adapters from external pipelines such as Hugging Face PEFT.

SageMaker AI benchmarks combinations of adapter count and concurrency and returns ranked results. The recommended deployment uses SageMaker Inference Components, with one component for the base model and one per adapter. That lets teams add, update or remove an adapter without disrupting the base model or other adapters.

A second September addition is an SLA-based concurrency search. Previously, teams picked concurrency levels by hand, ran a sweep, checked results against the service level agreement and repeated with a narrower range. Now users set search_recipe to max-concurrency-under-sla and supply targets for any combination of maximum TTFT, maximum time per output token, maximum end-to-end latency and maximum error rate.

AIPerf starts from a broad range and narrows it based on which probes pass or fail, which AWS says converges in fewer iterations than a linear sweep. The result reports the largest passing concurrency (feasible_max) or, if none qualifies, the smallest failing one (infeasible_min). Optional parameters bound cost and speed up convergence.

An agent skill for coding agents

In October 2026, AWS introduced the aws-ai-ml skill, available through the Agent Toolkit for AWS. It gives coding agents such as Kiro built-in SageMaker AI inference expertise. AWS argues that without grounded knowledge and access to real benchmarks, an agent can only guess, and a wrong instance choice shows up later as missed latency targets or wasted spend.

  • Find the right instance: ranked deployment options with measured throughput, latency and TTFT for a model in Amazon S3, SageMaker JumpStart or the Hugging Face Hub.
  • Benchmark a live endpoint: a report covering throughput, P50 and P99 latency, inter-token latency and supported concurrency.
  • Compare runs: the percentage change on every metric.

Answers are delivered as SageMaker Python SDK v3 code that engineers review and run under their own credentials. AWS says setup takes about 10 minutes, and the skill works with any agent that supports the Model Context Protocol (MCP) and in SageMaker AI Studio JupyterLab spaces.

What it means for teams running inference

In AWS's example, the support team replaces its manual April sweeps with one recommendation job covering all four adapters, followed by a concurrency search against a 500 millisecond TTFT target. The job can be started from a notebook or by asking a coding agent.

The post also begins a section on fitting more models onto each GPU, noting that Inference Components, which let several models share one endpoint and scale separately, reached the SageMaker AI console in October 2026. The source text is cut off at that point, so further details are not covered here.

AWSAmazon SageMakerAmazon SageMaker AI StudioNVIDIASageMaker JumpStartInference Gatewayenterprise AIcodingdata centers

Related news

AWS SageMaker AI Adds Monitoring for Inference Endpoint Drift
Products and toolsNovember 19, 2026, 22:28

AWS SageMaker AI Adds Monitoring for Inference Endpoint Drift

Amazon SageMaker AI can now identify latency drift, underused GPUs and missed serving optimizations on eligible generative AI inference endpoints. The monitoring is…

AWS Adds Cross-Region Inference Routing to SageMaker AI
Products and toolsNovember 19, 2026, 19:20

AWS Adds Cross-Region Inference Routing to SageMaker AI

AWS has introduced cross-Region inference routing in Amazon SageMaker AI, giving applications one invocation path across model endpoints and inference components in…

AWS adds GPU diagnostics and observability to SageMaker HyperPod
Products and toolsNovember 19, 2026, 17:41

AWS adds GPU diagnostics and observability to SageMaker HyperPod

AWS has introduced enhanced infrastructure observability in Amazon SageMaker HyperPod, bringing GPU diagnostics and profiling together for distributed training workloads…