Gantry · dataset evaluation · 13 September 2026

every episode moves one arm and holds the other still

Genesis-Intelligence/benchgen_lerobot_dataset_filtered, measured from the public release on 13 September 2026. frame level, every row read

Episode 2 of 1,275. The left arm is where it started.

Nobody opens a robot dataset and watches all of it. We read every frame of every episode and report what is in there.

One episode from the release, rendered in simulation rather than teleoperated. The right arm reaches in and works while every one of the left arm's seven action dimensions holds a single value from the first frame to the last. 699 episodes hold the left gripper at one value and 544 hold the right, so the release reads as single-arm demonstrations recorded in a two-arm action space.

episode 2 of Genesis-Intelligence/benchgen_lerobot_dataset_filtered, cut from the release's own video, 11 seconds. Nothing is re-rendered or sped up.

We read every frame of Genesis-Intelligence/benchgen_lerobot_dataset_filtered, 1,275 episodes and 268,822 rows. 699 episodes hold the whole left arm block at a single value for the entire episode and 544 hold the right, and the two sets do not overlap in the episodes we listed. That reads as single-arm demonstrations recorded in a two-arm action space, which is a reasonable way to build a benchmark and also means a policy is predicting seven constant channels in most episodes.

The release, as it actually is

propertyvalue
episodes1,275
frames268,822
frame rate20 fps
robotbimanual_marvin
action dimensions14
cameras3
distinct task strings1
shortest episode146 frames
median episode209 frames
longest episode661 frames

How each episode ends

A trailing hold is the number of frames at the end during which the commanded action does not change by more than 1e-4 on any component. Seconds come from the frame rate the release declares.

trailing hold at leastepisodesshare
1 frames , 0.05 seconds1,275100.0%
5 frames , 0.25 seconds65651.5%
10 frames , 0.5 seconds00.0%
30 frames , 1.5 seconds00.0%
60 frames , 3.0 seconds00.0%

How to check this without taking our word

The release is a meta directory plus one or more parquet files. Read data/chunk-000/file-000.parquet, group rows by the episode ranges in meta/episodes, and compute the same columns. No credentials.

What a full grading adds

This page is what a published release will tell you on its own. The full grading removes the part that scores worst and retrains on what is left, then reports whether the smaller corpus scores higher, with rollout counts and intervals. The one result measured on physical hardware is Stera-10M, two arms on a four arm SO-ARM101 bench, 21.9% against 36.9% task success across 1,270 rollouts.