Alibaba Cloud unveils AI storage lineup, from KV cache tier to 100 PiB CPFS

Why Alibaba Cloud is splitting AI storage by workload

Alibaba Cloud argues that AI workloads are not generic. In its guide for solution architects, the company says that training a trillion-parameter model, serving long-context inference, building a RAG pipeline and running thousands of agents each place different demands on throughput, latency, concurrency and data structure. Generic cloud storage, it says, forces compromises at every layer.

The announced portfolio is meant to remove those compromises by specializing storage for each stage. The source text lists five products: KVCacheStore, OSS Tables, CPFS, AgenticFS and enhanced EBS. This article covers the four that the available text describes in detail.

KVCacheStore: a new tier for inference

KVCacheStore is described by Alibaba Cloud as a dedicated storage tier for KV cache offload during LLM inference. It sits between GPU memory and remote shared storage, and the company calls it the first public cloud G3.5-tier KV cache storage product.

According to Alibaba Cloud, when KV cache exceeded GPU memory, the only options were token recomputation, which is expensive in compute, or remote shared storage, which is too slow for real-time inference. The company lists these figures:

• Support for hundred-billion-level KV cache entries
• 900% expansion of the cache coverage window
• 54% lower first-token latency on cache misses
• 50% lower per-token cost and a 99% effective cache hit rate when combined with Tair KVCM orchestration

The company recommends Tair KVCM as the orchestration layer with KVCacheStore as the G3.5 backend, particularly for context windows above 128K tokens, multi-turn agent sessions and high-concurrency real-time serving.

OSS Tables: objects, vectors and tables in one bucket

OSS previously supported only object storage. OSS Tables adds native vector indexing and similarity search, plus structured table storage with high-throughput point queries, inside the same bucket. Access to all three modalities goes through a unified API, with a single namespace and a single billing line.

Alibaba Cloud says this simplifies the standard RAG pipeline, which typically needs object storage for documents, a separate vector database for embeddings and a table store for metadata, linked by ETL pipelines. With OSS Tables, the company says, there is no vector database to provision, no ETL to maintain and no data duplication. Because embeddings live next to the documents they derive from, it also claims there is no sync lag.

The company adds a caveat: for advanced workloads that need complex hybrid search or fine-grained index control, a dedicated vector database may still be appropriate, and teams should evaluate case by case.

CPFS: parallel storage for training clusters

The new-generation CPFS (Cloud Parallel File System) targets large-scale AI training clusters, from hundreds to millions of GPUs, according to Alibaba Cloud. It is built on an end-to-end self-developed technology stack. The announced figures include:

• Aggregate throughput at the 100 TB/s level per filesystem
• Hundreds of millions of IOPS
• Single filesystem capacity of 100 PiB, a 5x increase
• 50% reduction in model startup time
• 30% increase in peak compute utilization
• 69% reduction in AI storage costs

Alibaba Cloud says the 100 PiB scale lets most training clusters work within a single namespace, so CPFS can serve as the one hot data layer for checkpoints, training data staging and intermediate results. In its view, the multi-tier staging pattern is no longer necessary.

AgenticFS and what it means for architects

AgenticFS is a file storage service in the NAS product family, designed for Agent Sandbox environments running on Container Compute Service (ACS) and Function Compute (FC). Alibaba Cloud calls it a brand-new product and the first file storage built specifically for agent sandbox workloads. The source text available for this article ends before the remaining details on AgenticFS and the enhanced EBS offering.

For architects, the pitch is consolidation and specialization: a dedicated cache tier for inference, one bucket for RAG data, and a single large filesystem for training. All performance and cost figures come from Alibaba Cloud itself, and the source does not include independent testing or pricing and availability details.