We fine-tuned the same policy twice on RoboArena DataDump 02-03-2026, once on all 9,589 rollouts as released and once with the 1,237 that spend a full cap of robot time and earn no credit removed, and the edited run scored 39.0% against 33.6%, a gain of 5.4 points over 1,200 rollouts.
A same-size random cut scored 33.9%, so the gain is from which rollouts were removed, not from removing rollouts.
Putting the marker in the jar moves most, and it and the strawberry instruction are the two the cut emptied hardest, taking 6 of 14 and 13 of 14 rollouts. Folding the towel and rotating the controller barely lost anything and do not move. The pooled lift clears zero, most single instructions at 120 rollouts do not, and the chart says so.
Measured on the release. Every rollout carries a duration in control steps and a partial_success score between 0 and 1, so each one reduces to a single number, the recorded seconds that earned no credit, which is duration over ten times one minus partial_success. The step rate is 10 Hz, confirmed by pulling eight wrist videos of four different lengths and finding every one of them runs at 10 fps with a frame count inside two frames of the recorded duration. The median rollout leaves 23.9 uncredited seconds. 1,237 of the 9,589 leave 39.9 or more, which is a full run of the 399 step cap with nothing scored on either outcome field, and that is 13.75 hours of the 94.97 hours of robot time in the release. The tail is a spike rather than a slope, since 1,228 of those 1,237 sit at exactly 39.90 seconds. It also lands on one policy, taking 534 of the 1,068 rollouts of paligemma_binning_droid against 8.2 percent across the seven other DROID policies it is ranked against. What stays is every rollout the evaluator gave credit to, including the 5,392 that also ran the cap out and still recorded a partial score, because a policy that reaches the drawer and fails to close it is a mistake the next model can see.
| Removed | rollouts | share of that site |
|---|---|---|
| Berkeley, spelled 15 different ways in the location field | 561 | 33.9% |
| frodobots, where one person scored every session | 428 | 11.9% |
| Stanford, Montreal, UPenn, NVIDIA, Washington, KAIST, Prague, Seoul | 211 | 8.9% |
| UT Austin and Yonsei | 37 | 1.9% |
| Removed | 1,237 | 12.9% |
| Kept | 8,352 | 87.1% |
Measured on the release. Each clip is the wrist camera of one rollout, the only view present for all 9,589 of them, cut from the stretch where the run was still going. The number in every caption is that rollout's uncredited seconds, recorded robot time that earned no score on either outcome field.