Auto focus
Published April 11 - 3 min read
Had some more fun along this vein again with more debug-visualizations in the live (non-release) @GrapicApp: https://t.co/BJ2jT5lJwd
— Nikolaus West (@NikolausWest) April 11, 2021
This time I took the first step towards taking control over camera auto focus. pic.twitter.com/gL9Q9ll1cy
For mobile #AR tracking to work in a variety of settings, you generally need some kind of auto focus. But the built-in algorithms obviously don't know the specifics of our use case. For us for instance, that means focus shifts away from the paper to your hand as you draw. pic.twitter.com/UlaU341Pa3
— Nikolaus West (@NikolausWest) April 11, 2021
I started out just looking at how the lens position (a value between 0 and 1 we get from iOS AVCaptureDevice) varies when auto focus is on. Also a good opportunity to make sure it doesn't vary when auto focus is turned off😛 pic.twitter.com/lmD29IPgAx
— Nikolaus West (@NikolausWest) April 11, 2021
I want to know when auto focus has converged but the isAdjustingFocus property on AVCaptureDevice doesn't seem to do the job during an ARKit session. Fortunately some simple heuristics based on the time between lens position updates did it since latency isn't critical here pic.twitter.com/MiADrUhDRn
— Nikolaus West (@NikolausWest) April 11, 2021
The next step was to pull in the rest of the state and estimations I thought might be useful to determine when auto focus should be on or off. In the process, I noticed that my background/foreground model seemed broken when the frame was partially out of view pic.twitter.com/vs2DQobg7E
— Nikolaus West (@NikolausWest) April 11, 2021
Since I have all my debug visualizations available live it was super easy to verify. There was clearly a bug in handling of edge pixels. It was a one line fix that still took an embarrassingly long time to find. pic.twitter.com/LbxKUJt764
— Nikolaus West (@NikolausWest) April 11, 2021
With that fixed I tried automatically turning off auto focus when the phone is still, nothing in the foreground, and auto focus converged, and turning it back on again whenever the phone has moved too far away along the camera axis. After some threshold tuning it seems to work pic.twitter.com/Xxar1eYGiT
— Nikolaus West (@NikolausWest) April 11, 2021
There is a lot more I'd like to do with camera focus but this was a nice small and fun way to get a better handle on the problem and still do something useful. Here is the final result as seen from the web viewer of @GrapicApp (coming in the next release) pic.twitter.com/n3PifXDMgV
— Nikolaus West (@NikolausWest) April 11, 2021