Apple paper proposes weighted state souping for long-context AI
A paper published on Apple’s machine learning research site proposes weighted state souping, a method designed to improve how state space models retrieve information from long, crowded contexts. The approach learns query-dependent weights for combining cached representations of documents, tools and memory records.
- Weighted state souping learns query-dependent weights for cached states.
- The method targets long and crowded contexts in state space models.
- Reported applications include document question answering, retrieval and tool use.
- A sparse variant offers near-constant encoding cost with respect to context length.
- The paper was accepted at the PALM Workshop at NeurIPS 2026.
A selective way to combine cached context
State space models, or SSMs, combine fast inference and a fixed state size with competitive language-modeling performance, according to the paper. State souping previously allowed an SSM to encode separate chunks of tokens independently, such as documents, tool definitions or memory records.
Those resulting state encodings can be cached and recombined to construct context for later queries. However, the paper says that both standard SSMs and state souping deteriorate when contexts become long and crowded. That limits their use in long-horizon domains.
Weighted state souping addresses this problem by learning weights that depend on the query. These weights determine how cached states are composed, allowing the model to give different importance to different pieces of stored context.
Reported gains across retrieval and tool use
The authors report that models using the method improve long-context performance across several tasks. These include single-hop and multi-hop document question answering, needle-in-a-haystack retrieval, variable tracking and tool use.
The method can also be adapted into a sparse variant. Apple’s paper says this version delivers near-constant encoding cost with respect to context length while preserving, and in some cases improving, performance.
The paper also presents theoretical arguments intended to illustrate why state souping works and when it may fail. It discusses the advantages of adding query-dependent weighting to the approach.
Relevance to agentic workloads
Weighted state souping combines reusable context representations with selective retrieval, according to the paper. The authors position it as an efficient approach to language modeling over large collections of documents, tools and other structured context.
That focus reflects the types of information used in the paper’s examples, including cached tool definitions and memory records. The work was accepted at the Workshop on Personalized, Aligned, Long-term Memory for AI Systems at NeurIPS 2026.
The paper is titled Weighted State Souping for Efficient Long-Context Retrieval. Its authors are William L. Tong, Aryo Lotfi, Emmanuel Abbe, Josh Susskind, Cengiz Pehlevan and Eran Malach. The paper identifies Tong as affiliated with Harvard University and notes that the work was done while at Apple.
