Google's latest flagships are here, and as with the previous two generations, the headline feature is the camera. Yet more computational magic has made its way into the camera on the Pixel 3 and 3 XL, including something called Super Res Zoom. It's practically the 'enhance' feature we've all been waiting for, but how does it work?
Pixel 2 v Pixel 3 at 7x zoom.
Ryan touched on this in his Pixel 3 review — he was pleasantly surprised by the results, even if they aren't quite as good as with optical zoom — and now a post on the Google AI Blog goes deeper into the mechanics of the feature. Previous attempts to better digital zoom have often rested on guessing missing details in a single cropped image, but Google's new method is a little more complex.
Digital photographs already use a process called demosaicing to fill in some blanks — as much as two thirds in some images — but digital zoom relies on this so much that the results end up not being so great. What Google does differently relies on something it already has good experience with: burst photography. It's used for the Pixel's best-in-class HDR+ algorithm, where it takes multiple sequential images and combines them to produce a better dynamic range.
Multiple images can be combined to fill in details.
With Super Res Zoom, what's important is that the burst images taken of a scene are from slightly varying positions. Natural hand tremor is usually enough to get shots that grab different details, but if your hands are too steady, the lens on the Pixel 3 camera will actually wiggle a bit to mimic this. With several images, even with just a pixel of movement in each one, the Google Camera app can use data from each to compose a photo with far greater detail than digital zoom has been able to achieve previously.
There are plenty of additional challenges involved in this process, which you can read about in Google's exhaustive blog post. It says that the results are roughly equivalent to photos taken using the 2x optical zoom lens on competing smartphones and it seems to be a pretty accurate claim, which is no mean feat.