Gantry · dataset evaluation · 13 September 2026

One task string covers all 55 episodes, and they range from 18 frames to 8,713

Written for Ugur Oezdemir at Bonsai Robotics. The dataset read here is apairo-robotics/tartandrive_offroad. It is not Bonsai Robotics's data, and it is on this page because TartanDrive, the public off-road driving corpus, the readable equivalent of the terrain Bonsai operates on.

55 episodes and 123,786 frames at 10 fps. One task string, drive off-road, covers all 55 episodes, so nothing in the release distinguishes a gentle track from a hard one. Episode length runs from 18 frames to 8,713, a range of nearly five hundred to one, which means episode count says very little about what the corpus holds.

episodes
55
123,786 frames at 10 fps
task strings
1
covering all 55 episodes
shortest episode
18 frames
1.8 seconds
longest episode
8,713 frames
871 seconds, 484 times the shortest

The release, as it actually is

Read from apairo-robotics/tartandrive_offroad. Where a figure came from the release metadata rather than the frames, the mode line below says so.

property value
episodes 55
frames 123,786
frame rate 10 fps
robot tartandrive
action dimensions 2
cameras 1
distinct task strings 1
episode length, shortest 18 frames
episode length, median 1,831 frames
episode length, longest 8,713 frames

Measurement mode, meta only, frame data is 4.1 GB and was not downloaded.

The result across eight corpora

The trailing hold tracks how the data was produced, not what the task was

A trailing hold is the number of frames at the end of an episode during which the commanded action does not change by more than 1e-4 on any component. Seconds come from the frame rate each release declares. Every corpus below was read the same way by the same code.

public corpus how it was produced episodes any trailing hold ten frames or more longest
lerobot/aloha_sim_insertion_scripted scripted sim 50 100.0% 86.0% 21 frames , 0.42s
RajatDandekar/so101_bimanual_napkin_fold real bimanual teleop 65 95.4% 75.4% 130 frames , 4.33s
weblucas/sponge_ball_2_yellow_tray_lerobot real single arm teleop 30 26.7% 6.7% 21 frames , 0.70s
lerobot/aloha_sim_insertion_human sim human teleop 50 6.0% 0.0% 1 frames , 0.02s
lerobot/ucsd_pick_and_place_dataset sim, fixed window 1,355 0.1% 0.0% 2 frames , 0.40s
RajatDandekar/eval_so101_pi0_bimanual_napkin_fold policy rollouts 6 0.0% 0.0% 0 frames , 0.00s

The ordering is the finding. A script stops commanding when it is done and the last rows repeat. A person keeps making small corrections, so a short take ends clean while a long bimanual take ends with the operator holding the fold in place. A policy rollout ends the moment the episode is cut. If somebody quotes you a single defect rate for idle frames across a mixed corpus, that number is averaging four different things.

How to check this without taking our word

Each corpus is a meta directory plus one or more parquet files. Read data/chunk-000/file-000.parquet, group rows by the episode ranges in meta/episodes, and count trailing rows where the action vector does not change. No credentials and no download beyond the frame data itself.

The wider record

What else has been graded

Thirty eight public robot datasets, one at a time. Each report measures one property across every episode, removes the part that scores worst, and retrains on what is left. The corpus figures and the cuts are computed from the releases. The lift column on the results page is read from each report's fine tune table rather than from a training run, and that page says so.

datasets graded
38
across 31 organisations
episodes examined
2,212,712
every one scored
episodes removed
123,143
5.6% of the total
rollouts graded
30,020
234 tasks

The one result measured on physical hardware is Stera-10M. Two arms on a four arm SO-ARM101 bench, identical per task fine tunes, 127 unpaired rollouts per cell, and the only difference between the arms is whether the human video pre-training stage happened.

Baseline
21.9%
no human video training. CI 18.8 to 25.3 · n=635
With ego stage
36.9%
after human video. CI 33.2 to 40.7 · n=635
Pooled lift
+15.0 pp
five tasks together. 95% range +10.0 to +19.8
Bench rollouts
1,270
5 tasks · 2 arms · 127 each

What grading Bonsai Robotics's own corpus would add

Everything above is a public release. The same four steps run on your data, and the first three cost nothing.

A few hundred episodes under whatever terms you want is enough for the first three.