Gantry · dataset evaluation · built 13 September 2026 · revised 14 September 2026

94% of episodes end on a held pose, and the spread says an operator put it there

DecisionFacts/Physical_AI_SO101_Cup_Nesting_Task, measured from the public release on 13 September 2026. frame level, every row read

What changed on this page, 14 September 2026

The first version of this page reported the trailing hold correctly and then drew the wrong conclusion from it. It said the rate “separates scripted takes from human ones”, which put this release on the wrong side of that line.

DecisionFacts reproduced the measurement independently, got the same numbers, and explained the cause. Their teleoperation protocol has the operator hold the final pose for a beat once the task is visually complete, as a manual task-complete signal before the episode is cut. That prompted a second look at the comparison set, and the original reading does not survive it. A single threshold share cannot tell an intentional hold from a scripted termination. The spread of the hold can, and on this release the spread says operator.

The measured figures below are unchanged, because nothing was wrong with them. The interpretation is replaced, and the new section that does the separating is Intentional hold or scripted termination.

We read every frame of DecisionFacts/Physical_AI_SO101_Cup_Nesting_Task, 200 episodes and 226,915 rows at 30 fps. 94% of episodes end with the commanded action no longer changing for ten frames or more, 98% for at least one frame, and the longest run is 6.2 seconds. Those are large numbers, and on their own they mean very little: a hold at the end of an episode is produced by a scripted terminator and by a careful human operator alike. What tells the two apart is not whether the hold is there, it is how much it varies, and on that measure this release does not look scripted.

The release, as it actually is

propertyvalue
episodes200
frames226,915
frame rate30 fps
robotso_follower
action dimensions6
cameras2
distinct task strings1
shortest episode671 frames
median episode1,111 frames
longest episode1,794 frames

How each episode ends

A trailing hold is the number of frames at the end during which the commanded action does not change by more than 1e-4 on any component. Seconds come from the frame rate the release declares.

trailing hold at leastepisodesshare
1 frames , 0.03 seconds19597.5%
5 frames , 0.17 seconds19296.0%
10 frames , 0.33 seconds18994.5%
30 frames , 1.0 seconds15678.0%
60 frames , 2.0 seconds7236.0%

Intentional hold or scripted termination

This is the question the first version of the page skipped. Both produce frames at the end where the commanded action stops changing, so a share at any single threshold cannot decide between them. The cleanest test available is a corpus released twice under the same task and the same robot, once driven by a script and once by a person. lerobot/aloha_sim_insertion is exactly that, and it is public.

releaseepisodesmedian holdlongest hold≥10 frames≥30 frames≥60 frames
aloha_sim_insertion_scripted50122186.0%0.0%0.0%
aloha_sim_insertion_human50010.0%0.0%0.0%
this release2005318694.5%78.0%36.0%

Read the last two columns, not the first. The scripted release holds on 86% of its episodes, so presence of a hold is not the signal. What it never does is vary: the median is 12 frames, the longest in fifty episodes is 21, and not one episode holds past 30 frames. That ceiling is the signature of a rule firing on a fixed condition. The human release of the same task barely holds at all, median 0 and a maximum of 1 frame, because nobody instructed those operators to wait.

This release sits nowhere near either. The median is 53 frames, the longest run is 186, more than three quarters of episodes hold past a second and over a third past two. A fixed termination rule does not produce a four-fold spread around its own median. A person holding a pose until they judge the task done produces exactly that, because the judgement takes a different length of time on every attempt. The measurement supports the protocol DecisionFacts described, and it is the variance rather than the share that carries the evidence.

For scale, the same read has now been run frame by frame over 36 public corpora, 22,430 episodes and 9,465,636 rows. Twenty of the 36 have a median trailing hold of zero. Eleven have a hold of ten frames or more on the majority of their episodes, so this pattern is common rather than exotic, and it tracks capture protocol far more than it tracks task. Only nine exceed a hundred frames at the maximum, and those are the ones worth asking a question about, which is what this page did.

One consequence worth stating plainly, because it is the part that matters to a buyer rather than to us. None of this is recoverable from the release itself. All 200 episodes carry a single task string, so a downstream user reading the files has no way to distinguish a deliberate hold-to-confirm from a stalled arm. The protocol is real; it just is not written anywhere the data can be read. A line in the dataset card fixes it.

How to check this without taking our word

The release is a meta directory plus one or more parquet files. Read data/chunk-000/file-000.parquet, group rows by the episode ranges in meta/episodes, and compute the same columns. No credentials.

What a full grading adds

Everything above is a read. It opens the released files, counts what is in them, and reports properties the release already has. It never trains anything, it costs minutes, and its ceiling is exactly what this page ran into: it can tell you an episode ends held, and it cannot tell you what the data teaches.

A full grading answers the second question, by training. A policy is trained on the corpus and a matched one without it, both are run on a physical bench, and the report is the difference in task success with its interval and its rollout count. Then the corpus is cut, the part that scores worst is removed, the training is repeated on what is left, and the report says whether the smaller corpus scores higher. That last step is the one a buyer cares about, because it converts “your data is fine” into “here is the subset worth paying for and here is what it is worth in points.”

The result measured end to end on physical hardware is the Stera-10M report: two arms on a four-arm SO-ARM101 bench, identical per-task fine-tunes, 21.9% against 36.9% task success across 1,270 rollouts. An example of the full deliverable, with illustrative figures, is linked below.