Gantry

Evaluation report, murobotics/tblock-all-piper-clean-v5-bi_piper_follower

We fine-tuned the same policy twice on murobotics/tblock-all-piper-clean-v5-bi_piper_follower, once on the release as published and once with the 90 episodes whose left gripper action never changes taken out, and the edited version scored 5.0 points higher across 700 graded rollouts.

Fine-tuned on full
22.9%
Stage success with all 176 episodes.
CI 19.9 to 26.1 · n=700
Fine-tuned on edited
27.9%
Stage success with the 90 flat left gripper episodes taken out.
CI 24.7 to 31.3 · n=700
Lift
+5.0 pp
From training on 51.1% fewer episodes.
95% range +0.4 to +9.5
Removed
51.1%
90 of the 176 episodes where the left_gripper.pos action holds one value from the first frame to the last.
21,820 of 47,539 frames · 12.1 of 26.4 minutes
Right gripper flat
0 of 176
Episodes where right_gripper.pos never changes. The narrowest right range anywhere in the release is 28.4 and the median is 40.9, against exactly 0.0 on 90 left channels.
left flat in 90 of 176 · right flat in 0 of 176

Success rate by task

fine-tuned on full murobotics/tblock-all-piper-clean-v5-bi_piper_follower fine-tuned on edited murobotics/tblock-all-piper-clean-v5-bi_piper_follower

A same-size random episode cut scored 23.1%, so the gain is from which episodes were removed, not from removing episodes.

Gain per task

The left gripper grasp gains most, which is the stage the removed episodes never demonstrate, since left_gripper.pos takes exactly one value in every one of them. Aligning and seating the block come out flat, and the chart says so.

What the edited version had removed

Measured on all 176 episodes and all 47,539 action rows of the release, read from the 176 data parquets and the 424 KB episode metadata file rather than a sample. Every episode gets one number, the range of the left_gripper.pos action from its first frame to its last, and that range is exactly 0.0 on 90 of the 176 episodes, 21,820 of 47,539 frames, so half a bimanual release commands the left gripper to a single constant for the whole take, 83 of them to 0.00 and the other 7 to a nonzero value they then hold. The right channel never does this once, its narrowest range anywhere is 28.4 and its median is 40.9. It is not the task being one handed. The right arm travels further than the left in 109 episodes, 19 of those 109 still work the left gripper through a median range of 23.5, and in all 67 left led episodes the right gripper keeps a median range of 35.5, so the idle side keeps its grip signal on the right and loses it on the left. It is not the hardware either, since only 1 of the 90 carries a nonzero bad_sensor_fraction.feetech. The release pipeline already writes this number down, in meta/source_quality.json, which records aperture_range_mm.left for every episode against an epsilon of 0.05 mm, sets reject_aperture_freeze to false, and reports 176 accepted and 0 rejected with zero findings on every episode. Our reading of the raw action columns agrees with that published field on 176 of 176.

Removedepisodesframesmedian left arm travel
left gripper on one value, left arm peak under 0.05 m/s while the right arm peaks 0.45 to 1.06266,465177
left gripper on one value, left arm still moving6415,355223
Removed9021,820206
Kept8625,719946

The removed media

Measured on the release. The six removed clips are cut from the packed 376 by 672 top camera at each episode's own from_timestamp, and the number in each caption is that episode's left_gripper.pos range, which reads 0.00 on every one of them while the right gripper in the same frames runs 38.2 to 65.8. The left gripper takes exactly one distinct commanded value across every removed episode, against a median of 60 distinct values on the ones that stay. The two kept clips are the release working properly, the widest left gripper range anywhere in it and the longest episode in the corpus.

episode_000000removed
The first episode in the release, and the left gripper holds 0.00 for all 319 frames while the right one opens and closes.
319 frames · left gripper range 0.00 · left arm peak 0.04 m/s against 0.91 on the right
episode_000020removed
Recording session tblock, where 26 of the 33 episodes carry a left gripper that never moves.
323 frames · left gripper range 0.00 · right gripper range 65.8, the widest of these six
episode_000071removed
Recording session tblock4_clean, the largest of the three, where 52 of the 116 episodes carry a left gripper that never moves.
321 frames · left gripper takes 1 distinct value · right gripper takes 138
episode_000162removed
The left gripper is parked at 0.63 rather than at zero and stays there for the whole take, so it is a constant and not a closed hand.
335 frames · left gripper range 0.00 · right gripper range 50.2
episode_000168removed
The left arm neither grips nor travels while the right arm does the whole task.
323 frames · left gripper range 0.00 · left arm peak 0.03 m/s against 0.75 on the right
episode_000171removed
The longest of these six clips, 12.6 seconds of one handed work published as bimanual.
377 frames · left gripper range 0.00 · left arm travel 213 against a kept median of 946
episode_000107kept
Kept, and the widest left gripper range anywhere in the release.
369 frames · left gripper range 79.78 across 101 distinct values · right gripper range 38.0
episode_000068kept
Kept. The longest episode in the release and one of only two past the 16.4 s duration fence, and the left gripper works the whole way through it.
736 frames · 24.5 s · left gripper range 72.67 across 191 distinct values
murobotics/tblock-all-piper-clean-v5-bi_piper_follower · 176 episodes · 47,539 frames · 5 stages × 140 rollouts × 2 arms Wilson 95% · cut = left_gripper.pos range exactly 0.0 across the episode