Google's focus on making Android more accessible has been evident — it opened last year's I/O keynote with morse code input on Gboard. In addition to that, Android Pie brought with it an array of new accessibility features, letting motor-impaired users easily jump to nine common interactions on their phones. Today, the Android Accessibility team announced Live Transcribe and Sound Amplifier, two new apps that look to materially improve quality of life for deaf and hard-of-hearing users. The apps are currently being tested on the Play Store and will be pushed to all Pixel 3 users in the next update.
While Automatic Speech Recognition (ASR) has been around for a while, it has grown increasingly sophisticated over time through deep learning. These improvements have culminated in Live Transcribe, a new accessibility feature that's powered by Google Cloud, letting hearing-impaired users follow along with a conversation happening around them in real-time. You can read about the finer nitty gritties of this technology over at the Google AI blog. The app has been developed with help from Gallaudet University, the world's only university designed around the needs of deaf and hard-of-hearing students. To give us some more insight, this video talks about the app's development and its effect on users' lives.
While the Live Transcribe page calls the app a closed beta and wants people to join a waiting list, we found and were able to install it from the Play Store. You can also grab it from APK Mirror – it should work on any device running Android 5.0 or newer.
Left: Users can follow along and type their own replies. Middle: Audio source is changeable. Right: Text size is adjustable, as is the theme.
For an app that's supposed to be in beta, it works impressively well. I had conversations around the phone, and the app was able to pick up most of what was said. Even when the transcription missed a word or two, you could get the gist of the conversation. The text size is adjustable, and there's a text box for you to type out a response and show others in the room. If the conversation stops for a while and then resumes, the phone can vibrate to alert you — I'm going to keep this enabled while working with noise-canceling earphones on. You can change the microphone source, even setting it to listen from a pair of headphones or a wearable device — has Google accidentally created a spying tool?
First mentioned at Google I/O 2018, Sound Amplifier works like a hearing aid, boosting the volume of conversations happening around you. In situations with a lot of ambient noise, the app can help you focus on quiet sounds happening near you, like someone's voice. The level of amplification is adjustable through sliders, and the feature can be enabled through the accessibility menu on your phone.
Live Transcribe is a useful tool for getting a written record of speech, but the spoken word only represents a portion of the sounds around us. Now Google's working to enhance the app's feature set by giving it the ability to recognize and describe sound events.
What's a "sound event?" Anything from a dog barking, to a car honking, to someone whistling as they walk by. Translate will soon identify these sounds and display them in a little notification separate from the speech transcription. Following up to a Twitter inquiry, Google even confirms that the service could be trained to recognize farts — so, lucky us, I guess.
In addition to transcribing sound events, the app is gaining the ability to let users easily copy text into other apps, or to just save a record of it for later. All this new functionality should be rolling out by mid-June.
With today's update to Live Transcribe, the aforementioned new features are now live — a good two weeks ahead of the mid-June date we were earlier promised! To enjoy this new functionality, make sure your app is up to date (V2.0.250643392), or download the latest version of Live Transcribe from APKMirror.
— Android (@Android) June 3, 2019