Apple Researchers Cut Search Agent Reasoning Latency by Over 90%
Apple researchers have proposed Selection-based Structured Reasoning (SSR), a framework in which small multimodal search agents choose their reasoning from pre-specified candidates instead of generating it freely. According to the paper, SSR reduces per-turn reasoning latency by over 90% while keeping task performance competitive with leading search agents of the same scale.
- Apple's SSR framework turns reasoning into selection from pre-specified natural-language candidates instead of freeform generation.
- Candidates are scored in parallel using teacher-forced prefilling and a shared context KV cache, without an auxiliary task head.
- Tested on seven multimodal search benchmarks with 2B and 4B models.
- Per-turn reasoning latency falls by over 90% and total per-question inference latency by 28, 54%.
- Average success rate is competitive with leading search agents of the same scale, according to the authors.
The problem: small models that think too much
Multimodal agents commonly generate freeform reasoning before each action. The Apple paper argues that this is a poor fit for small models. Because of their limited capacity, they can produce lengthy reasoning that offers little useful guidance for generating the next action, while still adding substantial inference cost.
The paper, titled "Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents," was published in October 2026 under the Methods and Algorithms research area on Apple's Machine Learning Research site. The authors include researchers affiliated with Carnegie Mellon University and the University of Illinois Urbana-Champaign, and some of the work was done while at Apple.
How SSR works
SSR reformulates reasoning as selection rather than open-ended generation. The framework represents recurring high-level reasoning as pre-specified, reusable natural-language candidates. At each turn, the model selects among these candidates based on their likelihoods given the current context. According to the authors, this is done without an auxiliary task head.
Because the reasoning traces are fixed in advance, the candidates can be scored in parallel. The paper explains that teacher-forced prefilling computes token likelihoods concurrently, both within and across candidates, using a shared context KV cache. In practice, this replaces a long sequence of generated tokens with a scoring step over a known set of options.
Results on seven benchmarks
The researchers evaluated SSR on seven multimodal search benchmarks using 2B and 4B models. They tested it across multiple reinforcement learning objectives as well as supervised fine-tuning.
- SSR achieves an average success rate competitive with leading search agents of the same scale, according to the paper.
- Per-turn reasoning latency falls by over 90%.
- Total per-question model inference latency drops by 28, 54%.
The authors summarize this as significant efficiency gains without sacrificing task performance.
What it means
The results point to a way of making compact multimodal agents cheaper and faster to run, since reasoning is often a major share of the cost per step. The reported numbers apply to the 2B and 4B models and the seven benchmarks tested, and the source does not describe results for larger models or deployment in specific products.
The work also sits alongside other Apple research on structured and efficient reasoning, including a July 2026 paper on learning structured reasoning via tractable trajectory control and a May 2025 paper on interleaved reasoning for large language models via reinforcement learning.
