Anonymous 3D markers
Reconstructing markers on the body without knowing which is which or their distances: epipolar lines, support, “one blob, one point”, tracking, implementation.
Transcript
1. Markers on the body
After calibration, the markers on the athlete. They are not on a rigid object: their distances change and are not known.
Here thirty 14 mm markers on a walking person, seen from the side and from above, with 8 cameras.
The body hides many markers in each camera. The problem is to reconstruct them in 3D without knowing which blob is which marker.
2. What two cameras see
The images of two cameras at the same instant. Blue dots: markers. Purple circles: two markers so close they merge into one blob. Red crosses: reflections.
No blob has a name. The possible combinations across 8 cameras with 20–30 blobs each are billions.
3. The epipolar line
Epipolar geometry narrows the search. A blob in the first camera defines a ray in space; that ray, seen from the second camera, is a line.
The right marker must lie on that line, within a tolerance of 1–2 pixels. The candidates light up: green the right one, red blobs that happen to be near the line.
Each candidate pair is triangulated: two rays give a 3D point. But two rays also cross by chance: some of these points are ghosts.
4. Support and “one blob, one point”
To tell true points from ghosts every candidate is projected into the other cameras: if there is a blob nearby, that camera confirms it. This is the support.
The right candidate is confirmed by 4 cameras; the wrong ones by none. Candidates with support of at least 3 are kept.
Then the “one blob, one point” rule: candidates with more support take their blobs; a candidate that wants blobs already used is discarded. In this frame: 243 candidates, 24 accepted points.
5. Tracking
From one frame to the next points are linked with a constant-velocity prediction and a Hungarian assignment within 4 cm.
A point is confirmed after two frames. If a track predicts a marker, support from 2 cameras is enough to recover it: useful when the body hides many.
6. Results
Recall versus number of cameras. With 4 cameras support 3 finds only a third of the markers; support 2 finds more but with 13% ghosts.
From 8 cameras the variants converge: 97% of markers, no ghosts, 1.1 mm error.
On the right the 3D error versus calibration error: with 0.05 degrees it doubles, with 0.1 degrees it triples. This is why the calibration of the first three papers is needed.
7. Implementation
In Python the algorithm takes about 58 ms per frame with 8 cameras: fine for offline analysis, not for real time at 120 fps.
The plan: blobs computed on the cameras, only coordinates on the network; candidates with an index along the epipolar lines; support and selection on a GPU.
Estimate: less than 2 ms per frame. And two indicators in continuous use flag a camera that has moved and needs recalibrating.