Gantry

Evaluation report, InternData-A1 ARX Lift-2

We fine-tuned the same policy twice on the ARX Lift-2 portion of InternData-A1, once on the release as published and once with 2,328 episodes removed, and the edited run scored 40.5% against 35.2%, a gain of 5.2 points over 1,200 rollouts.

Fine-tuned on full
35.2%
Task success with all 236,191 episodes.
CI 32.6 to 38.0 · n=1,200
Fine-tuned on edited
40.5%
Task success with the 2,328 repeated episodes removed.
CI 37.8 to 43.3 · n=1,200
Lift
+5.2 pp
From training on less data.
95% range +1.4 to +9.1
Removed
2,328
1.0% of 236,191 episodes, 2,835,808 of 163,766,023 frames.
26.3 hours of 1,516 · 13 of 382 subsets
The folding subsets
42
Distinct action records behind their 1,708 episodes, one of them stamped on 50.
1,708 head camera streams · all different

Success rate by task

fine-tuned on full InternData-A1 ARX Lift-2 fine-tuned on edited InternData-A1 ARX Lift-2

A same-size random cut scored 35.4%, so the gain is from which episodes were removed, not from removing episodes.

Gain per task

Fold short shirts, fold towels and the cloth task carry the gain, which is where the cut lands. Pouring water, whose subset the cut never touches, moves the other way. The pooled lift clears zero, most single tasks at 120 rollouts do not, and the chart says so.

What the edited version had removed

Measured on the release, every episode was reduced to one fingerprint, the mean, standard deviation, minimum and maximum of each of its commanded action channels, and compared byte for byte against every other episode in its own subset. Over 382 ARX Lift-2 subsets and 236,191 episodes, 2,328 episodes carry an action record that is bit identical to at least one other episode beside it. The 1,708 episodes of fold short pants, fold short shirts and fold towels hold only 42 distinct action records between them, one of them stamped on 50 episodes, and yet all 1,708 head camera streams have different image statistics, so the pictures move while the recorded motion repeats. Those three are also the only subsets in all 382 where every episode is exactly the same length, 1,200 or 1,900 frames, against a median of 152 distinct lengths in the 55 other subsets of comparable size, the tidiest of which still carries 79.

Removedepisodesdistinct action records
fold short pants, every episode exactly 1,200 frames69915
fold short shirts, every episode exactly 1,200 frames55914
fold towels, every episode exactly 1,900 frames45013
track the targets, records written twice and three times over486240
nine more subsets, every repeated record written twice13467
Removed2,328349
Kept233,863233,863

The removed media

Measured on the release. Each clip is the window that episode occupies inside its packed head camera mp4, read from the from_timestamp column of meta/episodes. The number in every caption is how many episodes in that subset carry the same action record, every mean, standard deviation, minimum and maximum bit identical on every commanded channel.

fold_towels/episode_000000removed
Fold towels
1,900 frames · 1 of 50 episodes carrying this action record · 13 records across 450 episodes
fold_towels/episode_000004removed
Fold towels, same action record as the clip above
1,900 frames · 1 of the same 50 · head camera statistics differ from every other episode
fold_towels/episode_000023removed
Fold towels, same action record again
1,900 frames · 1 of the same 50 · head camera statistics differ from every other episode
fold_short_pants/episode_000021removed
Fold short pants
1,200 frames · 1 of 50 episodes carrying this action record · 15 records across 699 episodes
fold_short_shirts/episode_000000removed
Fold short shirts
1,200 frames · 1 of 50 episodes carrying this action record · 14 records across 559 episodes
track_the_targets/episode_000192removed
Use the grippers to track the targets over the table.
793 frames · 1 of 3 episodes carrying this action record · 240 repeated records over 486 of the subset's 2,954 episodes
arrange_the_tableware/episode_000000kept
Setup the table, kept
722 frames · action record unique among 650 episodes · 137 distinct episode lengths in the subset
arrange_the_tableware/episode_000005kept
Setup the table, kept
758 frames · action record unique among 650 episodes · nothing in the subset repeats
InternData-A1 ARX Lift-2 · 236,191 episodes · 382 subsets · GR00T N1.6 · 10 tasks × 120 rollouts × 2 runs Wilson 95% · cut = an action record bit identical to another episode in the same subset