Gantry · dataset evaluation · 13 September 2026

Every one of the 1,355 episodes is exactly 50 frames long

Written for Jens Pommerening at Sereact. The dataset read here is lerobot/ucsd_pick_and_place_dataset. It is not Sereact's data, and it is on this page because a public pick and place corpus, the closest readable thing to the bin picking Sereact does.

1,355 episodes and 67,750 frames, and every single episode is exactly 50 frames, 100% of them at that length. That is a recording window rather than a task boundary, which decides what the last frames of an episode mean. Three task strings cover the corpus and the most common one accounts for 1,148 of the 1,355 episodes, 84.7%.

episodes
1,355
67,750 frames
all at one length
50 frames
100% of episodes, at 5 fps that is 10 seconds
distinct task strings
3
the largest covers 84.7%
episodes ending in a hold
0.1%
longest 0.40s

The release, as it actually is

Read from lerobot/ucsd_pick_and_place_dataset. Where a figure came from the release metadata rather than the frames, the mode line below says so.

property value
episodes 1,355
frames 67,750
frame rate 5 fps
robot unknown
action dimensions 4
cameras 1
distinct task strings 3
episode length, shortest 50 frames
episode length, median 50 frames
episode length, longest 50 frames

Measurement mode, frame level, every row read.

The result across eight corpora

The trailing hold tracks how the data was produced, not what the task was

A trailing hold is the number of frames at the end of an episode during which the commanded action does not change by more than 1e-4 on any component. Seconds come from the frame rate each release declares. Every corpus below was read the same way by the same code.

public corpus how it was produced episodes any trailing hold ten frames or more longest
lerobot/aloha_sim_insertion_scripted scripted sim 50 100.0% 86.0% 21 frames , 0.42s
RajatDandekar/so101_bimanual_napkin_fold real bimanual teleop 65 95.4% 75.4% 130 frames , 4.33s
weblucas/sponge_ball_2_yellow_tray_lerobot real single arm teleop 30 26.7% 6.7% 21 frames , 0.70s
lerobot/aloha_sim_insertion_human sim human teleop 50 6.0% 0.0% 1 frames , 0.02s
lerobot/ucsd_pick_and_place_dataset sim, fixed window 1,355 0.1% 0.0% 2 frames , 0.40s
RajatDandekar/eval_so101_pi0_bimanual_napkin_fold policy rollouts 6 0.0% 0.0% 0 frames , 0.00s

The ordering is the finding. A script stops commanding when it is done and the last rows repeat. A person keeps making small corrections, so a short take ends clean while a long bimanual take ends with the operator holding the fold in place. A policy rollout ends the moment the episode is cut. If somebody quotes you a single defect rate for idle frames across a mixed corpus, that number is averaging four different things.

How to check this without taking our word

Each corpus is a meta directory plus one or more parquet files. Read data/chunk-000/file-000.parquet, group rows by the episode ranges in meta/episodes, and count trailing rows where the action vector does not change. No credentials and no download beyond the frame data itself.

The wider record

What else has been graded

Thirty eight public robot datasets, one at a time. Each report measures one property across every episode, removes the part that scores worst, and retrains on what is left. The corpus figures and the cuts are computed from the releases. The lift column on the results page is read from each report's fine tune table rather than from a training run, and that page says so.

datasets graded
38
across 31 organisations
episodes examined
2,212,712
every one scored
episodes removed
123,143
5.6% of the total
rollouts graded
30,020
234 tasks

The one result measured on physical hardware is Stera-10M. Two arms on a four arm SO-ARM101 bench, identical per task fine tunes, 127 unpaired rollouts per cell, and the only difference between the arms is whether the human video pre-training stage happened.

Baseline
21.9%
no human video training. CI 18.8 to 25.3 · n=635
With ego stage
36.9%
after human video. CI 33.2 to 40.7 · n=635
Pooled lift
+15.0 pp
five tasks together. 95% range +10.0 to +19.8
Bench rollouts
1,270
5 tasks · 2 arms · 127 each

What grading Sereact's own corpus would add

Everything above is a public release. The same four steps run on your data, and the first three cost nothing.

A few hundred episodes under whatever terms you want is enough for the first three.