Anyscale on Azure Is Now Generally Available for Full AI Workflows

From renting tokens to owning the loop

Anyscale argues that paying per token for frontier model APIs is the fastest way to prove an AI use case, but that it comes with trade-offs. In its announcement, the company says teams give up control over three things: the model, their data and costs. The model roadmap belongs to someone else, data has to leave the company's environment to reach the model, and the bill rises in a straight line with usage.

According to Anyscale, AI-native companies hit these limits first. Their answer was to move to smaller, domain-specific models customized on their own data and run on their own GPUs, which the company says gave them faster responses at lower cost. The larger advantage, it says, came from continuously improving those models by owning the loop that turns production data into training and post-training runs, and then into updated inference deployments.

That loop replaces an inference-only pipeline with several workloads that need to share GPU clusters. Anyscale says Ray became the common runtime and orchestration layer for AI-native teams, and its platform turns Ray from a distributed compute engine into an enterprise-ready product.

How it runs on Azure

Azure customers deploy Anyscale from the Azure portal and run production AI on Azure Kubernetes Service (AKS) in their own subscription. They pay for it like any other Azure service, and spend counts toward their existing Microsoft Azure Consumption Commitment (MACC).

Anyscale says workloads run in the customer's subscription, associated with the organization's Microsoft Entra tenant, and that existing Azure RBAC and Azure Policy controls apply unchanged. Data, evals, models and container images stay in the customer's subscription. The network model is egress-only: every connection originates from the Anyscale operator and Ray clusters in the customer's AKS cluster and goes outbound to the Anyscale control plane, so no inbound firewall rules need to be opened.

The platform runs on open source Ray, which is governed by the PyTorch Foundation. Because Ray scales Python code, Anyscale says it works with frameworks teams already use, such as PyTorch and vLLM, and with new ones as they appear.

• Data curation for multimodal data such as text, images, video and documents
• Fine-tuning and post-training techniques including reinforcement learning (RL)
• Model serving through Anyscale Services, with zero-downtime upgrades, scale to zero when idle, and replica compaction to keep GPU utilization high

The company cites two examples. At Ray Summit 2026, Microsoft AI said its frontier reasoning model MAI-Thinking-1 was trained using Ray. BMW began using Ray for model training and later integrated it with its AI Gateway to run inference for open-source models across the enterprise.

Targeting GPU waste

Anyscale states that average GPU utilization on traditional ML infrastructure sits below 30% across the typical enterprise. By optimizing scheduling and orchestration for individual workloads and across teams, the company says it delivers 80%+ average GPU utilization, which it describes as 2-3x more work from the same GPU fleet. These are Anyscale's own claims.

At the workload level, the company says it spins up as many CPU workers as needed to load and preprocess data in parallel, so GPUs are not left idle waiting for the next batch. Fractional GPUs let teams pack more workloads or models onto each device. Autoscaling adjusts resources before a job runs out of memory, and checkpointing restarts a failed job from its last completed step rather than from scratch.

Across teams, Anyscale pools siloed clusters into shared compute managed from a single pane, drawing on spot, reserved and on-demand capacity. Scheduling is priority-aware: when an urgent job arrives, lower-priority work is paused rather than killed and resumes later. In the company's example, three teams share an H100 reservation, and the scheduler preempts a low-priority evals job when a high-priority training job lands. The scheduler can also place workloads across clusters and Azure regions, with Azure as the control point for identity and policy, and the same model can extend to GPU capacity a customer attaches beyond Azure.

What it means for enterprises

For enterprises already standardized on Azure, the release offers a route to run the full training-to-serving cycle under governance they already have, and to bill it against existing Azure commitments. Anyscale also points to a cost lever beyond utilization: post-training on a company's own data lets teams move to smaller, domain-specific models that cost less to serve.

The announcement quotes Microsoft Chairman and CEO Satya Nadella, who said earlier this year: "a company should be able to use a model without giving up the knowledge that makes it unique." Anyscale presents its Azure offering as a way to act on that idea.