Algorithm and checks
The algorithm step by step, reprojections and the filtering threshold, how many frames are needed, how to tell whether the calibration worked, the minimum error.
Transcript
1. The pipeline
The algorithm turns anonymous blobs into a calibration, in eight steps.
Blobs with a 40% threshold; static terna and initial camera estimate with a PnP; recognising the T in every image; shape filter for partially hidden blobs.
Then the bundle adjustment: cameras, distortion, delays and T positions estimated together. Rejection of observations with a high residual, multi-view relabelling, final bundle adjustment and validation.
2. Recognising the T
In every image the program tries all blob-to-marker assignments, also dropping one blob as a reflection, and keeps the best only if no other comes close.
If two solutions are similar the image is rejected: rejecting is safe, being wrong is not.
On the right the trade-off: with an ambiguity threshold of 2, 1.4% of images are wrong and 43% rejected. It takes 0.08 milliseconds per image.
3. Reprojections
After the bundle adjustment every estimated 3D point is reprojected into every camera and compared with the observed blob. The difference is the residual: here magnified up to 400 times.
Divided by the expected blob error it becomes the normalised residual. With noise only it would follow a Rayleigh distribution: 1.1% would fall beyond 3.
Here 8% falls beyond 3: partially hidden markers and, in the last bar, swapped markers, tens of standard deviations away. The filter discards observations beyond k = 3.
4. The filtering threshold
Which threshold k? Final error and wrong observations left, from k = 2 to no rejection.
After relabelling very few errors are left, below 0.1%, and the threshold changes the final error little.
k = 2 throws away 17% of good data. We choose k = 2.5–3: it removes gross errors at little cost.
5. How many frames
Left: error versus number of frames used, in a 40 second session. From 100 frames on the error stabilises around 0.65 mm.
Right: with the same 400 frames, the duration. 5 seconds give 1.7 mm, 20 seconds 0.63, 60 seconds 0.42. Volume coverage matters, not the number of frames.
The bundle adjustment with 8 cameras and 400 frames takes about 15 seconds in Python.
6. Did the calibration work?
In reality there is no truth. We simulate 90 sessions with random defects and look for quantities that predict the true error.
The reprojection residual does not work: many wrong calibrations have tiny residuals, because with little data the solver fits anything.
A validation wand does work: frames kept out of the solve, triangulated with the estimated cameras, comparing marker distances with the known ones.
No rule is perfect, but the wand rule never accepted a calibration with an error beyond 2.9 mm; a rule on the residual accepts some up to 10.6 mm. Better still: a second wand moved through the whole volume.
7. The error budget
Which minimum error can be reached? We start from the reference, 0.37 mm, and remove one error source at a time.
Blob noise does not matter. The unmodelled centroid error does: without it, 0.15 mm. 4K, 10 cameras and 60 seconds together: 0.28 mm; with an unmodelled error of 0.02 pixel: 0.10 mm.
Two things instead ruin everything: software synchronisation and an imprecise terna. To go lower: measure and reduce the unmodelled error, a 0.1 mm terna, a hardware trigger.