Evaluation report, Argoverse 2 Sensor

The same 3D detector fine-tuned twice on the Argoverse 2 Sensor validation split, once on the labels as released and once with the 80,172 cuboids whose dimensions are a repeated template rather than a measurement removed. Same 150 logs, same 23,547 frames, ten classes, 130 trials each. The edited run scored 5.8 points higher.

Fine-tuned on full

22.1%

Per-class detection score with every label kept.

CI 19.9 to 24.4 · n=1,300

Fine-tuned on edited

27.9%

Same 150 logs and same 23,547 frames, with the repeated-size labels dropped.

CI 25.6 to 30.4 · n=1,300

Lift

+5.8 pp

From training on less data.

95% range +2.5 to +9.2

Removed

4.7%

80,172 of 1,701,935 annotated cuboids, the labels on 1,856 of 18,395 tracks. No log and no frame was dropped.

1,856 tracks · 16 repeated sizes

One size, 1,427 tracks

1,427

Tracks carrying the identical cuboid 4.03 by 1.74 by 1.41 m, 7.8% of every track in the split, 1,426 of them labelled regular vehicle. Across a whole log their best lidar return has a median of 4 points, against 38 for every other track.

54,465 cuboids · median 1 return each · 37.6% hold none

Success rate by task

fine-tuned on full Argoverse 2 Sensor fine-tuned on edited Argoverse 2 Sensor

A same-size random cut of 80,172 labels scored 22.5%, inside the full arm's interval, so the gain is from which labels were removed and not from removing labels.

Gain per task

The class that gains most is regular vehicle, which holds 61,385 of the 80,172 removed labels. Bus and large vehicle sit flat, and between them the cut touches 307 labels. The pooled lift clears zero, most single classes at 130 trials do not, and the chart says so.

What the edited version had removed

Measured on all 150 logs of the released validation split, every annotated cuboid, no sampling. Cuboid size is fixed per track here, not one of the 18,395 tracks changes length, width or height across its log, and between them those tracks use 16,254 distinct sizes, which is what per-object measurement looks like. Sixteen sizes break the pattern. 1,856 tracks, 10.1% of the split, carry one of those sixteen repeated sizes, and 1,427 carry the same one, 4.03 by 1.74 by 1.41 metres. They are also the tracks the lidar barely sees, a median best return of 4 points across a whole log against 39 for every other track, and 34.0% of their individual boxes hold no return at all against 15.7% elsewhere. The rule never reads the lidar, so a thinly seen object whose size was measured for it stays, and so do the 3,076 frames where the rig sits under 0.5 m/s, because a stopped rig still watches a moving scene.

Removed labels, by class	cuboids	median lidar returns
Regular vehicle on a repeated size	61,385	1
Pedestrian on a repeated size	8,497	5
Wheeled device on a repeated size	4,526	3
Every other class on a repeated size	5,764	7
Removed	80,172	2
Kept	1,621,763	10

The removed media

Measured on the release. Each still is the ring camera frame nearest the lidar sweep that holds the box, taken from the camera whose arc the box falls in. The numbers are that box's exact size, how many tracks in the split carry the identical size, and how many lidar returns fell inside it.

dfc32963 ring_side_leftremoved

Regular vehicle, Pittsburgh, a crosswalk with cars parked down the block

size 4.03 by 1.74 by 1.41 m, shared by 1,427 tracks · 1 lidar return in the box · 11.9 m from the rig

29a00842 ring_rear_rightremoved

Regular vehicle, Pittsburgh, a commercial street beside a parked box truck

size 4.03 by 1.74 by 2.52 m, shared by 94 tracks · 0 lidar returns in the box · 9.4 m from the rig

19f53e16 ring_rear_leftremoved

Pedestrian, Washington DC, a bus shelter on a streetcar line

size 0.60 by 0.60 by 1.50 m, shared by 134 tracks · 3 lidar returns in the box · 9.1 m from the rig

070bbf42 ring_side_rightremoved

Wheeled device, Pittsburgh, a bank corner with people on the sidewalk

size 1.08 by 0.48 by 1.22 m, shared by 46 tracks · 5 lidar returns in the box · 9.1 m from the rig

a7636fca ring_side_rightremoved

Vehicular trailer, Pittsburgh, a fenced yard behind a wooden hoarding

size 3.41 by 2.50 by 2.33 m, shared by 23 tracks · 8 lidar returns in the box · 19.4 m from the rig

92b900b1 ring_front_rightremoved

Truck, Austin, a pickup parked against a warehouse wall

size 9.50 by 2.50 by 3.00 m, shared by 16 tracks · 5 lidar returns in the box · 14.9 m from the rig

b8489c02 ring_rear_rightremoved

Bicycle, Miami, a palm lined street outside a theatre

size 1.50 by 0.50 by 1.00 m, shared by 14 tracks · 2 lidar returns in the box · 10.7 m from the rig

7039e410 ring_front_rightkept

Regular vehicle, Detroit, a utility van working the kerb

size 5.43 by 2.79 by 2.52 m, unique to this track · 8,144 lidar returns in the box · 9.2 m from the rig

d5fa4d54 ring_front_rightkept

Pedestrian, Miami, people standing in front of a painted wall

size 1.11 by 0.60 by 2.13 m, unique to this track · 788 lidar returns in the box · 9.1 m from the rig

Argoverse 2 Sensor validation split · 150 logs · 23,547 annotated lidar frames · 1,701,935 cuboids · 10 classes × 130 trials × 2 arms Wilson 95% · cut = every cuboid on a track whose exact length, width and height is shared with four or more other tracks in the split