Live Caption was one of the highlight feature of Android 10 when Google unveiled it at Google I/O 2019. Its ability to generate accurate real-time captions regardless of audio source continues to blow us away — and it works offline, too. Google's Pixel phones were the first to get it, then it followed by expansion to other Android smartphones. Google announced yesterday that Live Caption is available to everyone using Chrome, and with some digging in the Chromium Gerrit, it looks like Chromebooks won't have to wait much longer to get it.
We've known fact Live Caption was making its way to Chromebooks since it first appeared in Chrome flags. We've been tracking its progress since February last year and kept a close eye on the Chromium Gerrit. Despite Live Caption's release in Chrome, Google plans to bind it to Chrome OS even tighter. Recently, we spotted a switch that enables it system-wide in Chrome OS Canary.
System-wide Live Caption in the Chromium Gerrit
According to the commit, Google is separating Live Caption from the browser on Chrome OS. Instead, it plans to add its files and SODA (Speech on-device API) within the filesystem. The lightweight model processes language in real-time and generates captions offline. With some effort, we've activated the experimental system-wide Live Caption on our Chromebooks.
Google has been working behind the curtains to add more languages in Live Caption. It's currently limited to English, but that may soon change in a future update. We can see a glimpse of Google's work by looking inside Live Caption in the OS settings accessibility page.
A list of languages to choose from in Live Caption settings
There's a brief explainer about Live Caption and how it'll add captions for media in Chrome. You're able to pick six languages: English, French, German, Italian, Japanese, and Spanish. Upon selecting a language, your Chromebook will download the language's speech files in seconds. Although it shows that it downloaded the files, Chrome OS doesn't yet properly add captions to foreign videos on YouTube. I've also tried playing media on Linux (Beta) and Android apps, but without luck.
Using Live Caption on Chromebooks
Live Caption on Chromebooks performs and functions about the same as its Chrome counterpart on the desktop. When transcribing, it adds a box towards the bottom of the screen and inserts words it picked up from the audio. The key difference between the two platforms is the way you activate the feature toggle: instead of placing the toggle in media controls, Chrome OS integrates it with the volume slider.
Live Caption toggle in Chrome OS.
Like Android, Chrome OS places Live Caption next to the volume slider for quick access. You can tap or click its toggle to switch the feature on or off. While nice that Live Caption is more accessible to users, I wish Google also made it available in the shelf's quick settings.
It'll likely take several weeks before Live Caption is formally available to use on Chrome OS, but we're actively looking forward to it, as it's great to see one of Android's best innovative features make its way to one of last year's most newly adopted platforms. We'll keep an eye out for changes in its development and update you on what we find.