Evaluation report, Menlo Robot Hand Poses
We fine-tuned the same vision to joint model twice, once on Menlo robot-hand-poses-train as released and once on an edited copy with the 2,293 poses no single tendon can hold removed, and the edited copy scored 5.5 points higher across 840 held out predictions.
Success rate by task
The seven groups are the joint families the model has to emit, the four tendon coupled finger pairs, the thumb chain lh_THJ5 to lh_THJ1, the two wrist joints and the four abduction joints, and a held out pose counts as a hit for a group when every joint in that group lands within 0.1 rad. A same size random cut scored 26.6%, so the gain is from which poses were removed, not from removing poses.
Gain per task
The groups that gain most are the coupled finger pairs, which is exactly where the cut lands. The wrist goes the other way and loses three, which is what a frozen output should do, since no cut can teach a model a constant it already emits. At 120 poses a group most single groups do not clear zero on their own and the chart says so.
What the edited version had removed
Measured on the release. The Shadow Hand drives 24 joints with 20 actuators, because four of them are fixed tendons, lh_FFJ0 lh_MFJ0 lh_RFJ0 lh_LFJ0, each pulling one finger's J2 and J1 together, so the split between those two joints is not a thing a controller can set. In the 11,000 poses we read the four pairs are labelled as if they were free, with correlations of -0.083, -0.083, -0.084 and -0.067 where one shared tendon would give close to +1. 2,293 poses, 20.8% of what we read, ask a single tendon to hold its two joints more than 1.20 rad apart, and the worst asks for 1.5638 rad, a middle knuckle at 1.5593 with the fingertip at -0.0045. Those are targets no hand can be scored against, and the rate in the released test split is 22.0%, so the eval carries them too.
| Removed | poses | worst split |
|---|---|---|
| one coupled pair commanded over 1.20 rad apart | 2,104 | 1.5638 rad |
| two coupled pairs commanded over 1.20 rad apart | 184 | 1.5272 rad |
| three coupled pairs commanded over 1.20 rad apart | 5 | 1.4970 rad |
| Removed | 2,293 | 1.5638 rad |
| Kept | 8,707 | 1.1999 rad |
The removed media
Measured on the release. Each pose in this release is one 1280x1280 render and there is no motion anywhere in the repository, so these are stills. Every caption carries that pose's own two coupled joint values and the split between them, in radians and in degrees.
