Artificial intelligence news.
Straight from the original announcements.

AWS adds three-tier Data Capture to SageMaker HyperPod

November 13, 2026, 19:01By the AI News Bites Editorial TeamSource: AWS
AWS adds three-tier Data Capture to SageMaker HyperPod
Image: AWS

Amazon SageMaker HyperPod now includes Data Capture, a built-in capability that records live inference inputs and outputs to Amazon S3. AWS says the feature is designed to help teams detect drift, identify failure patterns and improve models using production interactions.

Key points
  • Data Capture records live HyperPod inference data in Amazon S3.
  • Three independent tiers cover payloads, load balancer metadata and model-pod data.
  • AWS demonstrates Athena analysis, SQL labeling and fine-tuning in one cluster.
  • Tier 3 can capture full payloads for debugging and training-data collection.
  • AWS recommends encryption, sampling, lifecycle rules and PII redaction.

Three ways to capture inference data

HyperPod Data Capture provides three independent tiers, and teams can enable any combination in the same dataCapture block. Tier 1 is SageMaker endpoint capture, which stores full request and response payloads and is intended for compliance, Model Monitor and audit use cases.

Tier 2 captures Application Load Balancer access-log fields, including the request line, status codes, timings, client IP and user agent. AWS positions this tier for traffic and error-rate monitoring when teams only need metadata.

Tier 3 runs at the model pod. It stores full request and response payloads with event metadata and is intended for debugging, analytics and training-data collection. The feature uses an inference operator with NGINX, an OpenTelemetry collector and Amazon S3 uploader sidecars. AWS says Data Capture has minimal latency impact and zero impact on model availability.

A production-data flywheel on one cluster

AWS demonstrates the capability with a home-appliance support router using synthetic data. The walkthrough captures every request and response in Amazon S3, queries the records with Amazon Athena, labels them with a SQL policy, fine-tunes the router on the labeled data and redeploys the updated model alongside the original.

In the example, Athena surfaces a blind spot involving a new product. The fine-tuning job runs on a second GPU through the HyperPod training operator, while the inference operator serves the model on another node. The new model is then served by a second endpoint with capture still enabled. AWS says the example measures accuracy before and after fine-tuning on data that could not have existed before deployment.

The walkthrough uses Qwen2.5-3B-Instruct because AWS says it is small enough to fine-tune in about a minute and fits on a single NVIDIA L40S. The example requires a SageMaker HyperPod cluster orchestrated by Amazon Elastic Kubernetes Service, the inference and training operators, two available NVIDIA GPUs, kubectl and an Amazon S3 bucket in the same Region.

Controls for captured production payloads

Tier 1 and Tier 3 both store full payloads, so enabling both stores every record twice. AWS recommends Tier 1 for compliance and Model Monitor, Tier 2 when only traffic metadata is needed, and Tier 3 when teams intend to use payloads.

Because Tier 3 records can contain names, addresses and order numbers, AWS advises protecting the data before using it for training. Suggested controls include sampling, AWS Key Management Service encryption, Amazon S3 lifecycle rules and personally identifiable information redaction, for example with Amazon Comprehend.

In the deployment shown, Tier 3 is enabled with a 100% initial sampling percentage and captures both input and output, the defaults described by AWS. Teams can adjust the sampling percentage, buffer batch size, flush interval and maximum payload size.

Related news

AWS details four ways to speed up Amazon Bedrock AgentCore sessions
For developersNovember 13, 2026, 23:02

AWS details four ways to speed up Amazon Bedrock AgentCore sessions

AWS has outlined four architecture patterns for reducing session startup delays in Amazon Bedrock AgentCore runtime, including snapshot-based initialization, build hooks…

AWS Makes SGLang-Neuron Publicly Available for Trainium
Chips and infrastructureNovember 12, 2026, 23:03

AWS Makes SGLang-Neuron Publicly Available for Trainium

AWS says SGLang-Neuron is now publicly available through an AWS Deep Learning Container released by RadixArk in collaboration with Annapurna Labs. The plugin brings…

AWS Shows How to Build an Amazon Bedrock Tokenomics Data Lake
Products and toolsNovember 10, 2026, 17:56

AWS Shows How to Build an Amazon Bedrock Tokenomics Data Lake

AWS has published a design for a centralized observability data lake for Amazon Bedrock workloads, giving platform and machine learning teams a way to inspect model…