Gantry
Gantry · dataset evaluation · 18 September 2026

Four Starpilot releases carry one instruction string, the word manipulation

starpilot-ai/YAMGripper_Kitchen_FridgeFilling, measured from the public release on 18 September 2026.

instruction field carries no instruction

What is in the release

Every row is read out of the release. Nothing here is estimated and nothing is rounded up.

releases whose only task string is manipulation4
YAMGripper_Kitchen_FridgeFilling1 episode, 1 task string, manipulation
YAMGripper_Kitchen_FreezerFilling1 episode, 1 task string, manipulation
YAMGripper_BlackShirtFolding1 episode, 1 task string, manipulation
cell1 electronics packing unpacking8 episodes, 1 task string, manipulation
the task named in the repository name insteadfridge filling, freezer filling, black shirt folding, electronics unpacking
umiSample, by the same publisher31 episodes and 7 real task strings
those stringsArrange the counter, Set up the phone, Roll up the cable, Sort the paper stacks, Pencils in the cup, Pour the drink into the cup
releases excluded because they return HTTP 4011, cell1 blue short sleeve polo folding

Why this matters

The instruction is the one field a language conditioned policy reads, and in these four it is the word manipulation. The real task is sitting in the repository name, where no loader looks. umiSample shows the publisher already writes proper instructions, so this is a gap in the export path rather than a decision. Anyone pooling these with other corpora gets four sets of episodes that all claim to teach the same thing.

How to check this without taking our word for it

Read meta/tasks.parquet in any of the four and print the task column. One row, the string manipulation. Then read it in umiSample and get seven real instructions.

What a full grading adds

This page is what the release tells you on its own. A full grading removes the part that scores worst, retrains on what is left, and reports whether the smaller corpus scores higher, with rollout counts and intervals. The one result measured on physical hardware is Stera-10M, two arms on a four arm SO-ARM101 bench, 21.9% against 36.9% task success across 1,270 rollouts. The Stera-10M report.