Gantry · dataset evaluation · 13 September 2026

The human demonstrations hold still at the end and the policy rollouts never do

Written for Jason Ma at Dyna Robotics. The dataset read here is RajatDandekar/so101_bimanual_napkin_fold. It is not Dyna Robotics's data, and it is on this page because the nearest public human teleoperation corpus on the task DYNA-1 demonstrates, bimanual napkin folding, read alongside policy rollouts of the same task.

RajatDandekar/so101_bimanual_napkin_fold carries 65 human demonstrations of one task, 135,823 frames at 30 fps, every episode on the same task string. 95.4% of them end with at least one motionless frame and 75.4% with ten or more, the longest running 4.33 seconds. The same group published 6 policy rollouts of that identical task in RajatDandekar/eval_so101_pi0_bimanual_napkin_fold, and not one of those holds at all. So the hold is a property of how a human finishes a take, not of the task.

human demonstrations
65
135,823 frames at 30 fps
of those ending in a hold
95.4%
75.4% hold ten frames or more
longest hold found
4.33s
130 frames
policy rollouts holding
0%
of 6 rollouts on the same task

The release, as it actually is

Read from RajatDandekar/so101_bimanual_napkin_fold. Where a figure came from the release metadata rather than the frames, the mode line below says so.

property value
episodes 65
frames 135,823
frame rate 30 fps
robot bi_so_follower
action dimensions 12
cameras 3
distinct task strings 1
episode length, shortest 1,454 frames
episode length, median 2,033 frames
episode length, longest 3,094 frames

And the one it is compared against

RajatDandekar/eval_so101_pi0_bimanual_napkin_fold.

property value
episodes 6
frames 6,065
frame rate 30 fps
robot bi_so_follower
action dimensions 12
cameras 3
distinct task strings 1
episode length, shortest 201 frames
episode length, median 780 frames
episode length, longest 1,901 frames

Measurement mode, frame level, every row read.

The result across eight corpora

The trailing hold tracks how the data was produced, not what the task was

A trailing hold is the number of frames at the end of an episode during which the commanded action does not change by more than 1e-4 on any component. Seconds come from the frame rate each release declares. Every corpus below was read the same way by the same code.

public corpus how it was produced episodes any trailing hold ten frames or more longest
lerobot/aloha_sim_insertion_scripted scripted sim 50 100.0% 86.0% 21 frames , 0.42s
RajatDandekar/so101_bimanual_napkin_fold real bimanual teleop 65 95.4% 75.4% 130 frames , 4.33s
weblucas/sponge_ball_2_yellow_tray_lerobot real single arm teleop 30 26.7% 6.7% 21 frames , 0.70s
lerobot/aloha_sim_insertion_human sim human teleop 50 6.0% 0.0% 1 frames , 0.02s
lerobot/ucsd_pick_and_place_dataset sim, fixed window 1,355 0.1% 0.0% 2 frames , 0.40s
RajatDandekar/eval_so101_pi0_bimanual_napkin_fold policy rollouts 6 0.0% 0.0% 0 frames , 0.00s

The ordering is the finding. A script stops commanding when it is done and the last rows repeat. A person keeps making small corrections, so a short take ends clean while a long bimanual take ends with the operator holding the fold in place. A policy rollout ends the moment the episode is cut. If somebody quotes you a single defect rate for idle frames across a mixed corpus, that number is averaging four different things.

How to check this without taking our word

Each corpus is a meta directory plus one or more parquet files. Read data/chunk-000/file-000.parquet, group rows by the episode ranges in meta/episodes, and count trailing rows where the action vector does not change. No credentials and no download beyond the frame data itself.

The wider record

What else has been graded

Thirty eight public robot datasets, one at a time. Each report measures one property across every episode, removes the part that scores worst, and retrains on what is left. The corpus figures and the cuts are computed from the releases. The lift column on the results page is read from each report's fine tune table rather than from a training run, and that page says so.

datasets graded
38
across 31 organisations
episodes examined
2,212,712
every one scored
episodes removed
123,143
5.6% of the total
rollouts graded
30,020
234 tasks

The one result measured on physical hardware is Stera-10M. Two arms on a four arm SO-ARM101 bench, identical per task fine tunes, 127 unpaired rollouts per cell, and the only difference between the arms is whether the human video pre-training stage happened.

Baseline
21.9%
no human video training. CI 18.8 to 25.3 · n=635
With ego stage
36.9%
after human video. CI 33.2 to 40.7 · n=635
Pooled lift
+15.0 pp
five tasks together. 95% range +10.0 to +19.8
Bench rollouts
1,270
5 tasks · 2 arms · 127 each

What grading Dyna Robotics's own corpus would add

Everything above is a public release. The same four steps run on your data, and the first three cost nothing.

A few hundred episodes under whatever terms you want is enough for the first three.