SambaCloud adds prompt caching for MiniMax M3, cutting cached input cost 90%

What the feature does

Prompt caching was first introduced on SambaCloud with MiniMax M2.7 and is now available for MiniMax M3. When requests share a stable prefix of at least 4,096 tokens, SambaCloud can serve that prefix from cache instead of recomputing it. SambaNova says no code changes are required.

The feature runs on Automatic Prefix Caching. When the leading tokens of a new request match a prefix the system recently processed, those tokens come from cache, and only the new tokens, such as the latest user message or tool result, are processed fresh. The maximum cacheable length is 192,000 tokens. Under sustained traffic, SambaNova says hit rates typically climb above 90%.

The company ties the feature to the design of MiniMax M3, which it describes as built for long-horizon agents. Its 1M-token context window, powered by MiniMax Sparse Attention, lets whole repositories, multi-day agent logs and full research papers fit in a single request. SambaNova names three typical cases: a coding agent that loads a repository once and runs many edit-and-test turns, a research assistant that holds a set of papers in context, and a computer-use agent that carries the same instructions and tool definitions through a long task.

Latency and pricing figures

SambaNova reports that across 8k to 192k tokens of context, cache hits cut time to first token (TTFT) by 35% at the short end and 88% at the long end, with a median speedup of 4.7x. At 192k tokens, TTFT falls from 8.4 seconds to 1.0 seconds. The company's table gives these results:

• 8k: 0.52 s uncached, 0.34 s cached, 1.5x speedup, 35% reduction
• 32k: 1.38 s uncached, 0.46 s cached, 3.0x, 67%
• 64k: 2.71 s uncached, 0.58 s cached, 4.7x, 79%
• 128k: 5.42 s uncached, 0.81 s cached, 6.7x, 85%
• 192k: 8.36 s uncached, 1.04 s cached, 8.0x, 88%

On price, MiniMax M3 input costs $0.60 per million tokens at the standard rate and $0.06 per million for cached tokens. Output stays at $2.40 per million. The discount applies only to tokens actually served from cache, so savings depend on prefix length and how often requests hit the cache.

How to get cache hits and verify them

SambaNova advises putting stable content first: system prompt, documents, examples and tool definitions at the top, with changing parts at the end. The prefix must stay byte-identical. Rewording the system prompt, inserting a message before it or reordering content starts a new cache entry, while changing only the user message reuses the cached prefix. The first few requests populate the cache, and savings build as the prefix is seen repeatedly.

Every response includes a prompt_tokens_details object with two fields. cached_tokens shows how many tokens were served from cache and billed at the discounted rate. cache_creation_tokens is non-zero on the request that writes the cache entry and zero on later hits. It is informational and carries no extra charge. Subtracting cached_tokens from prompt_tokens gives the number of tokens processed fresh.

Limits and what it means for users

The company lists several caveats. Caching is available on MiniMax M3, and other models return cached_tokens: 0. Cache state is local to each serving node, so on multi-instance deployments a prefix is cached independently per node. Eviction follows a least recently used policy and depends on load, so there is no fixed time to live and persistence is not guaranteed. SambaNova also notes that actual gains depend on how often requests hit the cache.

For developers running agent loops that resend the same long context, the practical effect, per SambaNova, is that caching is already working on M3 without configuration. Checking prompt_tokens_details in the next response confirms it. Teams whose prompts change at the start of each request, or whose traffic is too sparse to keep prefixes warm, should expect smaller benefits. Full details are in SambaNova's prompt caching documentation.