Back in May, at Google I/O 2019, the company showed off a nifty new feature called Live Caption that would be able to transcribe content from media playback in near-real-time. We initially expected that it could land with Android 10, especially given the official Android site shows off the feature, but it's still missing as of today. However, based on some leaked APKs the folks at XDA Developers were able to get their hands on, it might debut on the upcoming Pixel 4. In fact, they even got it working on a Pixel 2 XL, and here's what it looks like in action:

Recording of Live Caption transcribing an unboxing video. Video via XDA Developers. 

A version of the Device Personalization Services app pulled from a Pixel 4 includes the software required for it to work (sort of like how a ton of features are baked into the seemingly-basic "Google" app), and with a bit of tinkering, it worked on an older Pixel 2 XL. Google previously said that the feature would be limited to higher-end devices as a result of resource requirements, but the Pixel 2 XL apparently has the requisite oomph.

Live Caption uses the new audio recording API in Android 10, but it won't work in a few scenarios, including voice and video calls, though XDA did test it as working for popular media apps including YouTube, Netflix, Amazon Prime Video, and even Google Photos and Google Podcasts.

Live Caption in Settings, toggle in volume control, and the Quick Settings tile. Images all via XDA Developers. 

Customization features for Live Caption include language settings (English-only, to start), a profanity toggle which censors output, and an option for a Live Caption toggle in the volume panel. It may also have a tile in the Quick Settings menu when enabled and can appear in both the Sound and Accessibility sub-menus in Settings.

Live Caption notification, first-time dialog, and caption in video while dragging down to dismiss. Images all via XDA Developers. 

When it's working and detecting audio, Live Caption has an ongoing notification which states that captions are enabled. You can disable Live Caption from that notification, or the box with the captions itself can be used to disable it by dragging it to the bottom of the screen, as with some other overlays like picture-in-picture videos.

Given that the software is baked into APKs included in pre-release Pixel 4 hardware, it sounds like we'll finally see the long-awaited feature land together with the upcoming phone, though who knows if or when older Pixels might get it, or where the cutoff in hardware requirements might place lower-end devices like the Pixel 3a or 2016 Pixels. And before you go asking "APK plz?" for your own Live Caption experiments, the folks at XDA have already stated you'll need more than just the leaked app to get it working.

Source: XDA Developers