Google's latest smart displays come with ultrasound sensing, a neat tech that uses ultrasonic waves to measure distance to nearby objects and adapt the UI depending on how near of far you are. Timers, weather reports, media controls, and other elements grow larger when you step away, to make it easier for you to spot them at a distance, then return to their regular size when you come close. This same technology appears to be at the heart of a new feature Google is working on, which proactively triggers Assistant the moment you come near without you having to say "Hey Google" first. Well, it's either this or the camera, but we're more inclined to believe it's using ultrasound.
Before you get too carried away, whether in geeky excitement or despair over the state of privacy in our homes, I need to make it absolutely clear that this feature isn't live yet on smart displays, and there's no telling when or if it will ever be. We're more inclined to believe it may never be released. What we do know is that Google is testing it internally on a dogfood firmware version and that it appears to work as intended there.
The video below comes to us courtesy of Jan Boromeusz, the same tipster who got the new Nest Hub UI earlier than everyone and gave us a sneak peek at its dark mode — both features were officially announced on Monday. For some mysterious reason, his Nest Hub Max runs a dogfood (internal, supposedly for Google employees only) firmware that's a few versions ahead of the public release. It also gets him access to a secret "Dogfood Features" menu in settings with a new "Blue Steel" option, which seems to be inspired by Zoolander (YouTube link).
When it's enabled, each time Jan walks in the room where his Nest Hub Max is, Assistant pops up and waits for his command. He doesn't need to say "OK Google" or "Hey Google" to trigger it. If he doesn't say anything, the prompt goes away. If he stays immobile, it also goes away, and he has to move closer to trigger it again. It's a bit magical, in a way, and I imagine it can speed things up if you're tired of repeating the keyword each time you want to ask for something.
Besides the uncertainty about this feature's public release (if ever), there are still a few things we don't know about this:
- Does the feature use the camera and facial recognition to trigger Assistant, the same way the Hub Max shows your personal cards when you come near it, i.e. Face Match? Or is it using ultrasound sensing? The fact that it works when Jan walks in the room before he even comes close tells us it's more the latter than the former, but we weren't able to verify it by closing the camera shutter and seeing if it still works. Jan told us his Hub Max lost connectivity shortly after tipping us and he can't get the display online anymore — cue in the conspiracy theories about Google remotely disabling that dogfood unit.
- Would Assistant get triggered if you wave your hand close to the Hub, similarly to how Soli works on the Pixel 4? That would be cool when cooking or doing chores. Again, Jan can't test it for us now.
A feature like this sounds like it'd be implemented for convenience's sake and at the expense of privacy. Assistant already gets triggered by sounds from the TV, any speech resembling "ey-oo-oo," and sometimes even random noises around the house. Imagine if it would start activating each time you came in a room with a smart display. In a way, that's very creepy, but we need to keep things in perspective.
First, you can always enable accessibility sounds that denote when Assistant starts and stops listening so you don't say something sensitive when it's on. (I recommend you turn these on regardless, they're handy in plenty of circumstances.) You can also disable ultrasound sensing or facial recognition, and close the camera shutter, to nip this feature in the bud. Finally, you can auto-delete your voice recordings every 3 months or choose to not save any recordings at all.
And well, if none of that sounds secure enough, you can always get rid of the smart display.