Gantry · dataset evaluation · 13 September 2026

1 of 200 episodes never actuate the gripper

VEXAutoSort/gear_v3, measured from the public release on 13 September 2026. frame level, every row read

The arm finishes the sort, then the recording keeps going

Nobody opens a robot dataset and watches all of it. We read every frame of every episode and report what is in there.

One full episode from a public sortation release. The commanded action stops changing before the episode ends. 62% of the 200 episodes end this way, and a policy learns that ending is part of the task.

episode 0 of VEXAutoSort/gear_v3, cut from the release's own video, 12 seconds. Nothing is re-rendered or sped up.

We read every frame of VEXAutoSort/gear_v3, 200 episodes and 55,544 rows. In 1 of them the gripper command never changes from the first frame to the last, so there is no grasp in those episodes at all. On a manipulation corpus that is the one episode type a policy cannot learn the task from.

The release, as it actually is

propertyvalue
episodes200
frames55,544
frame rate30 fps
robotso_follower
action dimensions6
cameras1
distinct task strings4
shortest episode48 frames
median episode274 frames
longest episode475 frames

How each episode ends

A trailing hold is the number of frames at the end during which the commanded action does not change by more than 1e-4 on any component. Seconds come from the frame rate the release declares.

trailing hold at leastepisodesshare
1 frames , 0.03 seconds18492.0%
5 frames , 0.17 seconds15376.5%
10 frames , 0.33 seconds12462.0%
30 frames , 1.0 seconds136.5%
60 frames , 2.0 seconds00.0%

How to check this without taking our word

The release is a meta directory plus one or more parquet files. Read data/chunk-000/file-000.parquet, group rows by the episode ranges in meta/episodes, and compute the same columns. No credentials.

What a full grading adds

This page is what a published release will tell you on its own. The full grading removes the part that scores worst and retrains on what is left, then reports whether the smaller corpus scores higher, with rollout counts and intervals. The one result measured on physical hardware is Stera-10M, two arms on a four arm SO-ARM101 bench, 21.9% against 36.9% task success across 1,270 rollouts.