Snorkel AI: Partial Credit Can Undermine Agent Training, Binary Gating Fixes It

The problem with all-or-nothing scoring

According to Snorkel AI, AI agents are increasingly trusted with long, open-ended work such as fixing bugs across large repositories, operating terminals for hours and chaining many tool calls toward one goal. The strongest models now solve a meaningful share of such tasks, but on the hardest ones failure is still the common outcome.

The standard post-training recipe is reinforcement learning against a verifier. The agent attempts a task many times, an automatic checker scores each attempt, and the model is pushed toward the best-scoring attempts. When success is rare, this runs into a basic problem: if the checker reports only whether the task was fully solved, almost every attempt on a hard task scores zero, and a group of identical zeros teaches nothing.

The natural remedy is partial credit, which scores each attempt by the fraction of requirements it met. Snorkel AI says this can work against its own goal. A model trained on it can learn to satisfy the easy checks while giving up on the route that satisfies them all.

Binary and fractional returns

The post describes a verifier that runs M binary checks once an attempt is over, as with a test suite for a coding agent or a checklist for a tool-use agent. The deployment goal is assumed to be that every required condition is satisfied. Snorkel AI defines B as the binary all-pass return and R as the terminal fractional return, a weighted fraction of passed checks. Because every weight is positive, B equals 1 exactly when R equals 1, so the fractional return keeps the all-pass outcome and can also separate failures from each other.

The authors call both quantities returns rather than rewards. In reinforcement learning, they write, a reward belongs to a single transition, while a verifier scores a finished trajectory as a whole. They describe calling such scores rewards as a "textbook mistake".

A trajectory that passes nine of ten checks and one that makes no progress both receive B = 0. That suits the deployment objective, the post says, but it discards information that may predict how to reach completion. Proposition 1 in the post states that fractional return can carry information about the completion value of intermediate decisions that pass/fail labels cannot represent. The authors stress this is an information result, not a guarantee that pushing a policy toward higher fractional return helps completion.

Two ways GRPO can go wrong

The study focuses on GRPO (Shao et al., 2024), which samples a group of rollouts per prompt and labels each by how its return compares with the rest of the group. Snorkel AI identifies two failure modes.

• Within a group: fractional return can order trajectories differently from completion. After centering, a failed rollout can receive a positive label and be reinforced even when a successful rollout is in the same group.
• Across groups: the same prompt can yield a mixed group on one draw and an all-fail group on the next. A rule that switches to fractional return only on all-fail groups changes what training optimizes, even if every individual group looks fine.

The post also says that blending fractional return with the pass/fail signal does not remove the within-group conflict, since a failed attempt can still be reinforced alongside a successful one unless almost all the weight sits on pass/fail.

Binary-gated GRPO and what it means for practice

Snorkel AI introduces binary-gated GRPO, which it says removes the within-group conflict while keeping what partial credit offers. The authors say that whether partial credit then helps or hurts completion comes down to a single quantity that the scores alone cannot reveal. With binary-gated GRPO, they write, harm requires a specific combination of conditions, and in a natural setting partial credit can only help.

The post says its central results do not depend on the particular form of fractional return. Binary gating, what the all-fail fallback optimizes and the deciding quantity apply unchanged to any score assigned to a whole trajectory, such as a sum of per-turn scores. The authors also say they close with a practical rule for real training runs, and that proofs are in an appendix.

For teams training agents on hard, long-horizon tasks, the message is that adding partial credit is not automatically safe, and that how it is combined with the pass/fail signal matters. The excerpt provided does not include the final rule or any experimental results.