We've known for a while that Google was interested in making its own chips. Recently their hardware efforts even bore some fruit, but most of us assumed that it would be quite a while until Google made anything at a consumer level.
Well, it turns out that the new Pixel 2 phones have a bit of Google-designed hardware inside them. It's called Pixel Visual Core, and it brings even better camera performance to the phones — or it will, once it's turned on. Pixel Visual Core augments Google's HDR+ implementation, improving latency, power efficiency (using 1/10 the power), and speed (5x faster). Later this year third-party developers can even use it to bring HDR+ photography to their own applications. PVC is also programmable, so additional functionality utilizing this chip could be enabled by Google in the future.
The star of the show is Google's Image Processing Unit (IPU) with eight Google-designed custom cores. The specialized hardware is further improved by Google's usual software skill, allowing for a greater degree of integration and control. The IPC also supports Halide and TensorFlow, which should both make using it easier for developers, and can expand the potential use cases.
Developers should get their first taste of the Pixel Visual Core via the developer preview for Android 8.1 Oreo, apparently as soon as "the coming weeks." Later on it'll be enabled for all third-party apps that use the Android Camera API.
Third-party app photos, no HDR+ (left) and HDR+ (right)
Up until now, if you wanted HDR+ photos on Google's hardware, you had one choice: the stock Google Camera app. But this hardware will let 3rd party camera applications use HDR+, meaning soon you could be Instagramming or Snapchatting Pixel HDR+ photos directly from the viewfinder of your choice instead of having to go through the process of capturing a proper photo using the camera app and then selecting the photo in another app.
On an unfortunate note, this bit of hardware isn't currently available to any applications on the Pixel 2 or 2XL. I don't just mean third-party apps, even the stock Google Camera app can't use it yet. Google says that they'll enable it sometime in the coming months. Soon, though, Google will enable an option in the developer settings of the new Pixels to turn on the PVC, allowing developers to tinker with the functionality and test it in their applications. Presumably, this means that we'll be seeing some apps on the Play Store with beta 3rd-party HDR+ support for Pixel 2 in the near future.
If you're interested in seeing how it all works, as we are, you'll probably have to wait until then. Google's been pretty mum on the details so far, but if you want to learn more you can check out Google's blog post on the subject.
Alternate title: There's a surprise present hiding in the Pixel 2, and we aren't even allowed to unwrap it yet.