Evaluation report, aloha_pen_uncap_diverse
We fine-tuned the same policy twice on physical-intelligence/aloha_pen_uncap_diverse, once on the release as published and once with the 19 episodes whose fixed 6.00 second window ends in the longest motionless hold taken out, and the edited version scored 5.3 points higher across 1,000 graded rollouts.
Success rate by task
A same-size random cut scored 37.1%, so the gain is from which episodes were removed, not from removing episodes.
Gain per task
The pooled gain clears zero with a low end of +1.0 points. Two of the five stages come out flat on their own at this trial count, and the chart says so.
What the edited version had removed
Measured on the release. All 123 episodes are exactly 300 frames, one bucket in the length histogram, so every take was cut to the same 6.00 second window at 50 fps, and pen uncapping does not take 6.00 seconds every time. After the last commanded motion the arms sit still, and across the release those tails run to 2,262 frames under 0.005 rad of joint motion per frame, 45.2 seconds of held pose that a policy is trained to imitate. The cut takes the 19 episodes whose hold runs 0.80 seconds or longer, 15.4% of the release, and those 19 carry 42.4% of every held-still frame in it. The 36 episodes still moving when the window ends are kept, including the one with the least joint travel in the whole release, because a demo cut off mid-motion is something a policy can learn from and a held pose is not.
| Removed | episodes | held-still frames |
|---|---|---|
| Hold of 1.26 to 1.32 s | 3 | 192 |
| Hold of 0.96 to 1.18 s | 10 | 512 |
| Hold of 0.80 to 0.90 s | 6 | 254 |
| Removed | 19 | 958 |
| Kept | 104 | 1,304 |
The removed media
Measured on the release. Each card is the cam_high frame at 5.90 seconds, 295 frames into the fixed 300 frame window. The number is how much of the end of that window the arms spend motionless.
