Gantry · full grading · example report

What the data taught, on a physical arm

An example of the full grading deliverable, section for section, so you can see what arrives before commissioning one.

Read this first, this is an example

Every figure on this page is illustrative. Nothing here is a measurement of any real corpus, and nothing should be quoted as a result. The tasks are labelled A to E because in a delivered report they are your task strings.

What the example does show honestly is the method, the section order, the statistics and the standard of evidence. Every interval is a Wilson 95%, every rate carries the count it came from, and the limits section is the one a delivered report actually carries. That is the part worth looking at.

This page is still being worked on. For a full grading with real numbers and real names in it, the published one is the Stera-10M report.

Baseline18.5%task fine-tune only
CI 15.6 to 21.9 · n=610

With the corpus31.1%corpus stage, then the same fine-tune
CI 27.6 to 34.9 · n=610

Pooled lift+12.6 pp95% range +7.7 to +17.5

After curation35.4%worst-scoring 19% removed, retrained
CI 31.7 to 39.2 · n=610

Bench rollouts1,2205 tasks × 2 arms × 122

Dose responser² 0.86gain against matching hours
1.0 would be perfect

What was measured, and against what

Two arms. Identical per-task fine-tunes on both. The only difference between them is whether a pre-training stage on the corpus happened first. 122 unpaired rollouts per cell across five tasks, so 1,220 attempts on physical hardware. Nothing in this report comes from simulation, and nothing comes from a validation loss.

This matters more than it sounds. Fine-tuning a large model on almost any data moves its numbers, so a report that shows only that the model improved after training on your corpus has not established that your corpus did anything. The comparison arm exists so that the number the report pays out is a difference, not a level.

The corpus as received

Read frame by frame from the release, before anything was trained. This table is the read, and it is the part a customer can reproduce in an afternoon.

property	value
episodes	1,412
frames	1,913,640
frame rate	30 fps
action dimensions	6
cameras	2
distinct task strings	5
median episode	1,284 frames
hours after non-manipulation removed	41.6 h
hours the arm could execute	33.8 h

The last two rows are the ones customers are most often surprised by, and they are reported before any training so the headline corpus size is never mistaken for the size that trained. A capture that is genuinely 1,400 episodes can be 30 usable hours, and saying so is what makes the rest of the report defensible.

Per task, with every interval

Tasks are labelled A to E here. In a delivered report these are your task strings. A task counts as moved when its whole 95% range sits above zero. Four of five do here. The fifth is reported, not dropped.

task	matching hours	baseline	with corpus	lift	moved
Task A	18.0	26.2% 32/122 · 19.2–34.7	47.5% 58/122 · 38.9–56.3	+21.3 +9.4 to +32.5	yes
Task B	9.5	21.3% 26/122 · 15.0–29.3	33.6% 41/122 · 25.9–42.3	+12.3 +1.3 to +23.0	yes
Task C	7.2	18.0% 22/122 · 12.2–25.8	29.5% 36/122 · 22.2–38.0	+11.5 +1.0 to +21.7	yes
Task D	6.0	14.8% 18/122 · 9.6–22.1	26.2% 32/122 · 19.2–34.7	+11.4 +1.4 to +21.2	yes
Task E	0.9	12.3% 15/122 · 7.6–19.3	18.9% 23/122 · 12.9–26.7	+6.6 −2.5 to +15.6	no
pooled	41.6	18.5%	31.1%	+12.6 +7.7 to +17.5	yes

The gain is not evenly earned

Each task gain tracks how many hours of matching footage the corpus held for it, r² 0.86. Task A had 18 hours and gained 21 points. Task E had under an hour and did not move at all.

This is the single most useful number in the report and it is the reason the engagement tends to repeat. It turns a backward-looking result into a forward-looking decision: what you capture next is predictable from what the last capture bought you. A corpus owner who knows the slope can spend the next three months of capture on the tasks where the slope is steep, rather than on the ones that are already saturated.

Where attempts stopped

Binary success discards most of what a rollout tells you. The operator instead logs how far each attempt got along the declared ladder for the task, so a failure has a location rather than just a zero.

no contact → approach → contact → grasp → transport → success

In this grading most of the gain is attempts that previously died at approach and now reach grasp: 31% of baseline attempts stopped at approach against 16% after the corpus stage, while attempts reaching grasp rose from 20% to 29%. That is a mechanism rather than a delta, and it is frequently the part a customer acts on, because it says which phase of the task the footage is actually teaching.

It is also what makes small corpora reportable at all. Two datasets can be indistinguishable on binary success and clearly separated on the rungs, and a report that only counted successes would have called them equal.

The curation result

This is the step that separates a grading from a certificate. Every episode is scored, the worst-scoring portion is removed, the training is repeated on what is left, and the same bench protocol is run again. If the corpus is uniformly useful the smaller version scores worse. If it is not, the smaller version scores higher, and the difference is the cost of the part that was carried along.

corpus	episodes	hours	success	vs full corpus
no corpus stage	—	—	18.5%	−12.6
full corpus	1,412	33.8	31.1%	—
curated, worst 19% removed	1,144	27.1	35.4%	+4.3

Six and a half fewer hours, 4.3 points better. Stated honestly, the 95% range on that gain is +0.2 to +8.5, which clears zero and only just. A delivered report says so in exactly those words rather than rounding the caveat away, and where the range does not clear zero the report says the curation did not pay and explains what would be needed to decide it.

The removed episodes are listed by id in the delivered version, with clips, so the finding can be inspected rather than believed.

What this grading could not conclude

This section is in every delivered report. A grading that only reports what worked is a marketing document.

Task E is unresolved, not negative. With 122 rollouts per cell and under an hour of matching footage, a real gain of six points is indistinguishable from none. Deciding it needs either more footage on that task or roughly four times the rollouts, and the report says which is cheaper.
Two of the five tasks share a scene, so their results are not fully independent and the pooled interval is very slightly optimistic. Quantified in the delivered version.
The protocol is unpaired. A paired design would tighten every interval materially, and needs the same scene reproduced twice on the bench, which costs setup time rather than money.
Training loss is reported and is not evidence. It is included because it gets asked for. This project has already published the case where loss anti-predicted capability, which is why the payout is bench success.
This measures what the corpus teaches this policy class on this robot. A different robot with a different reach envelope pulls a different fraction of the corpus through, and the report says what that fraction was.

What it needs from you, and what comes back

item	detail
From you	read access to the release, the task list, and whatever you already know about how it was captured. Gated or private is fine.
Time	about three weeks end to end, most of it bench time rather than compute
Back to you	this document with your names in it, a public URL if you want one, the episode-level scores as a file, and the removed-episode list with clips
Repeatable	the same protocol on your next release, so the two reports compare directly and you can show a trend rather than a snapshot

The sections above are the default. They are not fixed. If the number your buyers ask for is per-operator, per-site, per-session or per-task, the report is cut that way instead. If what you need is a single defensible figure for a paper, that is a different and shorter document. Tell us what the report has to prove and to whom, and it gets built to that.

A real one, with real numbers All 38 gradings