Google's AI projects are by far some of the most impressive in the world. If you use a Pixel device, there's AI behind almost everything you do, from taking photos to talking to Assistant. Google's latest experiment asks you to lipsync with a popular song, and in return, you get a scorecard rating your efforts. Think Singstar without the music videos or the ability to choose the song, and you've got an accurate idea of what this does.
It really is that limited, only featuring one small segment of one song — "Dance Monkey" by Tones and I. That means you can see everything LipSync has to offer in one go. We don't know if Google plans to add more songs or not, but either way, LipSync feels underwhelming for now. Despite its limited nature, this is still something everyone should check out.
LipSync doesn't record any audio. It bases its scoring solely on the movement of your lips, thus the name. Google is teaching its AI to learn how humans talk by monitoring the way our faces move. The fact this is possible without any specialist motion tracking hardware — it only uses your camera — shows how far machine learning has come in the last few years.
But why should you care? How is Google learning to understand human speech without audio going to benefit you? For most of us, it won't. But for the thousands of people living with ALS and other illnesses impacting speech, it could mean everything.
As Google explains once you've finished the experiment, there are a lot of people out there unable to do something we take for granted — speaking normally. Whether it's ALS or some other condition, there are thousands of people in this world struggling to be understood. Google is working on a system that can track facial movements of people with similar conditions and decipher what they're trying to say. Once it's figured that out, it can speak it out loud on the person's behalf.
So by participating in this project, we can help Google develop new ways to help people, and it only takes a few minutes. You can try LipSync for yourself via the source link below.