Apple's RLTL;DR Lets AI Learn From Its Own Failures on Tasks It Cannot Solve

Apple researchers have introduced RLTL;DR, a training method that lets a model improve on tasks it almost never solves by writing short feedback on its own failed attempts. According to the paper, a Qwen 3.5 9B Thinking policy that stayed flat at a Pass@1 of 0% to 1% under standard GRPO training reached 14, 31% with the new approach.
- Apple's RLTL;DR has a model write a single TL;DR insight after each failed attempt and condition later rollouts on those insights.
- On tool-calling and coding tasks filtered to Pass@128 = 0, standard GRPO on Qwen 3.5 9B Thinking stays at 0% to 1% Pass@1.
- RLTL;DR reaches 14, 31% Pass@1 with insights in context, and 12, 13% with no insight in context at eval time.
- The SFTL;DR variant, trained on only 4k (task, insight) tuples, recovers almost the full performance of RLTL;DR and classical SFT on full rollouts.
The problem with learning from rare successes
The paper, published in October 2026 and tied to NeurIPS, starts from a limitation of reinforcement learning with verifiable rewards (RLVR). In this common setup, an agent makes multiple attempts at a task and the training optimizes toward the successful ones.
According to the authors, this becomes a problem in self-improvement settings. There, tasks can be so difficult that the agent has a low or even no chance of success. There are also no teacher models or example solutions to distill from. If almost every attempt fails, there is little signal to learn from.
How RLTL;DR works
In RLTL;DR, the policy is shown the verifier outputs after each failed attempt and then writes its own feedback in the form of a single TL;DR insight. The next rollout is conditioned on all previous insights, and rollouts are sampled sequentially until a solution is found.
The authors add a second step: they enable backpropagation on the in-context insights. The goal is to internalize a direct task → insight mapping, so that the model carries the lesson without needing it spelled out in the prompt.
The paper is by Michael Kirchhof, Eleonora Gualdoni, Andrew Szot, Khashayar Gatmiry, Aryo Lotfi, Abbas Kazerouni, Omar Attia, Sanjoy Chowdhury and Alexander Toshev.
Results on hard tool-calling and coding tasks
The evaluation used challenging tool-calling and coding datasets, filtered to Pass@128 = 0. On these, standard GRPO training of a Qwen 3.5 9B Thinking policy stays flat at a Pass@1 of 0% to 1%, according to the paper.
RLTL;DR, the authors report, breaks through this learning barrier. It achieves a Pass@1 of 14, 31% with insights in context during training. The authors stress that it also reaches 12, 13% when no insight is in context at evaluation time.
They identify the task → insight internalization as the key ingredient.
SFTL;DR: a compact training variant
To study this further, the team reduced the approach to SFTL;DR. It trains only on (task, insight) tuples, without showing or backpropagating on any rollouts. According to the paper, training on only 4k of these tuples recovers almost the full performance of RLTL;DR and of classical SFT on full rollouts.
The authors say this points to a promising compacted training paradigm of the form "on this sort of task, keep this sort of thing in mind", and they hope it inspires future research.
What it means
The results suggest that when a model has no teacher and almost never succeeds, its own condensed feedback on failures can still provide a usable learning signal. The reported gains are measured on a specific model and on datasets filtered to be very hard, so the paper does not by itself show how the method performs elsewhere. The finding that short insights can stand in for far larger sets of full rollouts is the part the authors single out for follow-up work.


