Last week I posted a teardown of the Settings app from XE 16. That was really just a teaser, because this update is huge. This time I'm ripping into the guts of every app on Glass. That's over 100 apks (counting multiple version updates from the last month), and there's plenty to see. I'm not going to waste time on a long intro, you know what you're here for!

New APKs

One of the biggest changes to hit Glass with the update to XE 16 can be seen without even cracking open any apks. The total number of apps went up from 44 to 52. We lost 3 apks, but 11 completely brand new packages joined the ranks.

• Bluetooth.apk (this is in addition to GlassBluetooth)
• Board_id.apk
• FusedLocation.apk (this appears to replace the NetworkLocation.apk and uses the FusedLocationProvider)
• GlassShare.apk
• GlassSync.apk
• GmsCore.apk
• InputDevices.apk
• Keyguard.apk
• SharedStorageBackup.apk
• Shell.apk
• TelephonyProvider.apk

Most of these are fairly anorexic in terms of unique content and functionality, and a couple even look more like placeholders for future content. Regardless of their current size and function, they are new and demonstrate that the developers are working to modularize some of the core components of Glass.

More About Languages

The Glass Voice apk has been rearranged a bit and it now includes new asset folders for the US English and Japanese languages. Not only does this further confirm efforts to prepare Glass for new markets, but it's also strong evidence that Japan will be the first destination outside of the US.

Things get more interesting inside of these folders. Each one contains a hotword_paths.config file which holds the key to explaining one of the new requirements for 3rd-party app developers. As of XE 16, official Glassware are required to use only voice triggers provided by the GDK. Keep in mind, this requirement only applies to the voice triggers on the "Ok, Glass" screen, this doesn't prevent apps from transcribing voice commands once they are running.

This change was made to remove the requirement that trigger commands make a round trip to Google. If simple voice processing occurs locally, it reduces lag and data usage. The hotword_paths.config file contains phonetic mappings for each of those built-in phrases.

Note: There were some unrecognizable characters separating phonetic variants. I've replaced them with an '*' so they will show up.

[hotword_google]*g uw g ax l*g uw g l

[hotword_take_a_picture]*t ey k ax p ih k ch er*t ey k ax p ih k sh er*t ey k ey p ih k ch er*t ey k ey p ih k sh er

[hotword_record_a_video]*r eh k er d ax v ih d iy ow*r eh k er d ey v ih d iy ow*r ih k ao r d ax v ih d iy ow*r ih k ao r d ey v ih d iy ow

[hotword_get_directions]*g eh t d ih r eh k sh ih n z

[hotword_send_a_message]*s eh n d ax m eh s ih jh*s eh n d ey m eh s ih jh

[hotword_show_me_transit_times]*sh ow m iy t r ae n s ih t ay m z*sh ow m iy t r ae n z ih t ay m z

[hotword_make_a_call]*m ey k ax k ao l*m ey k ey k ao l

[hotword_make_a_video_call]*m ey k ax v ih d iy ow k ao l*m ey k ey v ih d iy ow k ao l

[hotword_record_my_score]*r eh k er d m ay s k ow r*r ih k ao r d m ay s k ow r

[hotword_reply_to_all]*r ih p l ay t uw ao l*r iy p l ay t ax ao l

Remember when I said there was a folder for English and Japanese? There are a total of 127 phrases mapped out in English. Japanese, on the other hand, has one... Yeah, so Japan might still have to wait a little while.

[hotword_ok_glass]o: k e: g u r a s u

New Voices

So far, everything we've seen has been about recognizing a user's speech, but Glass also talks back to people in the right circumstances. A synthesized voice has to come across in a language users can comprehend and in an accent that isn't too foreign. That's where the GoogleTTS apk comes in. Glass has always shipped with a female voice with a western US accent, but there are about to be several new options and a few new languages.

<string name="voice_pack_status_gender_female">Female</string>
<string name="voice_pack_status_gender_male">Male</string>
<string name="voice_pack_status_updatable">Update available</string>
<string name="voice_pack_status_downloadable">Download (~%1$s)</string>
<string name="voice_pack_status_locale_default">Current voice</string>
<string name="voice_pack_status_downloading">Downloading...</string>
<string name="voice_pack_name">%1$s</string>
<string name="voice_pack_name_high_quality">%1$s (high quality)</string>
<string name="voice_pack_description">"Name: %1$s
Locale: %2$s
Data size: ~%3$s
Speaker gender: %4$s"</string>
<string name="voice_pack_action_download">Download (~%1$s)</string>
<string name="voice_pack_action_remove">Remove</string>
<string name="voice_pack_action_update">Update (~%1$s)</string>
<string name="voice_pack_action_locale_default">Make it current voice for %1$s</string>
<string name="voice_pack_action_cancel">Cancel download</string>
<string name="voice_pack_install_tab_installed">INSTALLED</string>
<string name="voice_pack_install_tab_all">ALL</string>
<string name="voice_pack_install_tab_settings">SETTINGS</string>
<string name="voice_entry_details">Launch voice details dialog</string>
<string name="voice_entry_overflow_desc">Additional voice pack actions</string>
<string name="voice_entry_download_desc">Download voice pack</string>
<string name="voice_entry_cancel_download_desc">Cancel voice pack download</string>
<string name="voice_entry_delete_desc">Delete voice pack</string>
<string name="voice_entry_sample">Play voice sample</string>
<string name="voice_pack_not_enough_space">"Not enough free space on internal storage. You'll need to free some space to install this voice."</string>
<string name="action_settings">Options</string>
<string name="action_licenses">Open Source licenses</string>
<string name="action_voice_pack_update">Update</string>
<string name="action_voice_pack_uninstall">Uninstall</string>
<string name="voice_pack_uninstall_dialog">Uninstall this voice?</string>
<string name="voice_pack_uninstall_keep">Keep</string>
<string name="voice_pack_uninstall_uninstall">Uninstall</string>
<string name="voice_pack_unpack_failure">Failed to install the downloaded voice</string>
<string name="multi_voice_pack_title">%1$s voices</string>

Language packs will be downloadable on demand and they will have friendly "names" (eg. Sue, Amy, Brian). There are also some sample .ogg files already on Glass for each of the options, but no built-in way to listen to them. They all sound about as good as the default voice, but still obviously synthetic. The really cool part is that these look like the same downloadable packages already available for Google's TTS engine. Here's the full list:

English (United Kingdom)
en-GB (female) - https://dl.google.com/dl/android/tts/v2/en-gb-x-fis-phone-hmm-r2.zvoice
en-GB (female) - https://dl.google.com/dl/android/tts/v2/en-gb-x-fis-diphone-usel-r2.zvoice
en-GB (male) - https://dl.google.com/dl/android/tts/v2/en-gb-x-rjs-phone-hmm-r2.zvoice
en-GB (male) - https://dl.google.com/dl/android/tts/v2/en-gb-x-rjs-diphone-usel-r2.zvoice

English (United States)
en-US (female) - https://dl.google.com/dl/android/tts/v2/en-us-x-sfg-phone-hmm-r2.zvoice
en-US (female) - https://dl.google.com/dl/android/tts/v2/en-us-x-sfg-diphone-usel-r4.zvoice

German (Standard)
de-DE (female) - https://dl.google.com/dl/android/tts/v2/de-de-x-nfh-phone-hmm-r2.zvoice

Portuguese (Brazil)
pt-BR (female) - https://dl.google.com/dl/android/tts/v2/pt-br-x-afs-phone-hmm-r2.zvoice

Spanish (Spain)
es-ES (female) - https://dl.google.com/dl/android/tts/v2/es-es-x-ana-phone-hmm-r2.zvoice

Spanish (United States)
es-US (female) - https://dl.google.com/dl/android/tts/v2/es-us-x-sfb-phone-hmm-r2.zvoice

French (France)
fr-FR (female) - https://dl.google.com/dl/android/tts/v2/fr-fr-x-vlf-phone-hmm-r2.zvoice

Italian (Italy)
it-IT (female) - https://dl.google.com/dl/android/tts/v2/it-it-x-kda-phone-hmm-r2.zvoice

Korean (Korea)
ko-KR (female) - https://dl.google.com/dl/android/tts/v2/ko-kr-x-ism-phone-hmm-r2.zvoice

There are two interesting things about this list. The first is that there are only 2 male voices, and both are British. Observant readers may have also noticed that Japanese is missing entirely. For those counting, that makes 3 different apks with some degree of support for new languages, but there isn't a single language that is covered in all of them. In fact, only Japanese and German appear more than once. Maybe the next update -whenever it comes- can clear things up a bit.

Bluetooth File Transfers

Despite a fairly significant dependence upon Bluetooth, Glass doesn't actually do very many things with it. The only two common activities are data tethering with a handset and using the hands-free profile for voice calls. There's now a bit of evidence that Glass may eventually allow files to be sent directly between it and other devices.

Before I continue, I suggest mentally filing this one as fairly doubtful. The evidence comes from the new Bluetooth apk and several strings look more appropriate for phones and tablets. At this point, it's equally likely that these strings were left over from a standard Bluetooth package. Still, it would be pretty great to be able to send pictures directly from Glass to a computer or a friend's phone with a couple of quick taps.

<string name="incoming_file_confirm_title">File transfer</string>
<string name="incoming_file_confirm_cancel">Decline</string>
<string name="incoming_file_confirm_ok">Accept</string>
<string name="incoming_file_confirm_timeout_content">There was a timeout while accepting an incoming file from "%1$s"</string>
<string name="incoming_file_confirm_Notification_title">Bluetooth share: Incoming file</string>
<string name="incoming_file_confirm_Notification_caption">Do you want to receive this file?</string>
<string name="incoming_file_toast_msg">Incoming file from another device. Confirm you want to receive this file.</string>
<string name="notification_receiving">Bluetooth share: Receiving %1$s</string>
<string name="notification_received">Bluetooth share: Received %1$s</string>
<string name="notification_sending">Bluetooth share: Sending %1$s</string>
<string name="notification_sent">Bluetooth share: Sent %1$s</string>
<string name="app_name">Bluetooth Share</string>

New Icons

We already hit quite a few new icons in the Settings app, but there are a few more that show up in almost every apk on the system. A few of these are obviously there for system functions, and the rest are probably included for use by 3rd-party app developers.

Location Sharing

I missed calling this out in the previous article because I assumed there were only icons for location sharing. However, when I stumbled onto a few strings that have been around for a while, I realized the icons actually complete the package of resources. Assuming this isn't a coincidence, it looks like Glass Maps will be able to send either current locations or destinations to other people.

<string name="sharing_no_share_targets">Visit google.com/myglass to add friends</string>
<string name="sharing_menu_share_confirming">Sharing</string>
<string name="sharing_menu_share_confirmed">Shared</string>
<string name="sharing_menu_send_confirming">Sending</string>
<string name="sharing_menu_send_confirmed">Sent</string>
<string name="sharing_menu_tap_to_annotate">Tap to add caption</string>
<string name="sharing_menu_ok_glass_or_tap">Tap or say "ok glass" for options</string>
<string name="sharing_menu_item_add_caption">Add caption</string>
<string name="voice_menu_item_add_caption">add a caption…</string>

There has also been a bit of movement on the code, but nothing that clearly indicates that this could be turned on tomorrow. Still, I wouldn't be surprised to see location sharing in the next month or two.

Glass Home

There were a lot of interesting hints hidden away in the GlassHome apk, the majority of which have to do with voice commands.

A Full List Of Built-In Voice Commands

I mentioned the new policy of pre-specified voice triggers in the section about languages, and Google provides a list of these phrases. The thing is, there are actually several more phrases already supported by Glass, but they haven't been made official in the GDK. Some of these are responses or follow-up statements instead of trigger phrases, but they are still supported by built-in speech recognition.

Note: There are 127 phrases in hotword_paths.config and only 91 in this list. The disparity comes from redundant phrases that have identical meanings but different wording.

 

 

Each trigger also includes a set of strings as feedback depending on the success or failure of the action.

<string name="voice_menu_item_find_a_dentist">find a dentist</string>
<string name="voice_menu_context_item_find_a_dentist">Find dentist</string>
<string name="voice_menu_disambiguation_find_a_dentist">find a dentist with…</string>
<string name="voice_menu_temporary_message_find_a_dentist">Finding</string>
<string name="voice_menu_confirmation_message_find_a_dentist">Found</string>

Among these strings are also some for disambiguation. They will be used to clarify which app should be activated to complete a given command. For example, a person might say "find a doctor," after which they would be prompted, "find a doctor with..." if more than one app supports the phrase. Acceptable responses might be something like "Yelp" or "Google Maps." This functionality resembles Android's sharing intents which give users a chance to pick from a list of applications for different types of actions. The Glass team would eventually have to offer something like this as users started to install more apps with similar activities, like those for mapping, social networks, or fitness.

Control Music With Your Voice

Having Play Music on Glass is definitely cool, but it's a bit disenchanting to start your songs with a voice command and then have to resort to the touchpad for any subsequent action. This oversight is about to be corrected. There is a new menu and matching set of voice commands that reflect typical player controls. By the way new music player code is sprinkled throughout the apks, it looks like 3rd-party music players will also be able to use this functionality in the future.

** New Voice Commands **

<string name="musicplayer_voice_menu_play_music">Play music</string>
<string name="musicplayer_voice_menu_pause_music">Pause music</string>
<string name="musicplayer_voice_menu_next_track">Next track</string>
<string name="musicplayer_voice_menu_previous_track">Previous track</string>
<string name="musicplayer_voice_menu_start_radio">Start radio</string>
<string name="musicplayer_voice_menu_start_instant_mix">Start instant mix</string>
<string name="musicplayer_voice_menu_exit_music">Exit music</string>

** musicplayer_voice_menu.xml **

<menu xmlns:android="http://schemas.android.com/apk/res/android">
<item android:id="@id/musicplayer_voice_menu_play_music" android:title="@string/musicplayer_voice_menu_play_music" />
<item android:id="@id/musicplayer_voice_menu_pause_music" android:title="@string/musicplayer_voice_menu_pause_music" />
<item android:id="@id/musicplayer_voice_menu_next_track" android:title="@string/musicplayer_voice_menu_next_track" />
<item android:id="@id/musicplayer_voice_menu_previous_track" android:title="@string/musicplayer_voice_menu_previous_track" />
<item android:id="@id/musicplayer_voice_menu_start_radio" android:title="@string/musicplayer_voice_menu_start_radio" />
<item android:id="@id/musicplayer_voice_menu_start_instant_mix" android:title="@string/musicplayer_voice_menu_start_instant_mix" />
<item android:id="@id/musicplayer_voice_menu_exit_music" android:title="@string/musicplayer_voice_menu_exit_music" />
</menu>

Google Services Framework

Just like Glass Home, there are a couple of interesting bits hidden here.

More New Icons

I'll start with some icons that don't appear in any of the other apks. Except for the focus icon at the end, they are all scaled to the smallest size Glass apps use, so they are meant for fairly compact placement in the UI. None of them are exciting by themselves, but a few of them might make sense with the next part.

^{Note: The icons with 'dark' in the name also have 'light' counterparts, which are actually dark grey in color.}

Are We Getting Toast Notifications?

If there were ever a simple feature that just seemed to be missing from Glass, it has got to be toast notifications. There are just some things that don't belong as a card in the timeline, either because they aren't important enough or won't be relevant for long. However, users should still see things are happening, and that's what toast messages are great for. In all likelihood, these would only show up when the display is already on, and they might show simple things like connections to wifi networks or inform users of a new message when they are in the middle of some other activity.

My evidence is a new layout named toast_notification.xml, and it looks like the real deal. The layout defines a single row with an icon on the left followed by some text to the right. The background is set to be specifically 100dp (150px) tall, so about 40% of the screen would be covered. The background color is set to black with 53% opacity, and the text is white with full opacity. I hope the background becomes more opaque in the future, because that transparency allows way too much to be seen through it (see the mock-up below).

<color name="toast_notification_background">#881f1f1f</color>
<color name="toast_notification_text_color">#fff0f0f0</color>

Some of the icons above happen to be well suited to this particular application, so it does make sense. There is even some code that references the standard Android Toast class, but the layout itself isn't actually used anywhere yet. In other words, like many other new things in XE 16, it's still a work in progress.

^{Approximation of toast messages on Glass.}

The Other Stuff

There are quite a few other puzzle pieces within the GoogleServicesFramework that could suggest other improvements and features, but many of them don't make enough sense without more context. Four layouts caught my attention because they only appear in this apk, but explaining their presence is a little tricky, so I'm just going to leave it to readers. They are:

• inline_keyboard.xml
• keyboard_keys_layout.xml
• playback_controls_header.xml
• playback_controls_view.xml

The playback control views aren't referenced in code yet, so they might have been placed there by accident. They are likely connected to the music player voice menu discussed in the section for GlassHome, but I'm not sure why the two components wouldn't be in the same apk. The keyboard layouts actually do show up in code, but I'm not positive how and when that code will be called. Whatever their purpose, these layouts could be a part of some pretty cool features down the road.

Wrap-Up

Despite fewer user-facing features than expected after the Glass team took a 4 month hiatus from updates, it's clear that there really have been a lot of things happening in the background. Most of these changes and improvements still seem incomplete, but we can probably expect many of them to turn up over the next couple of months, assuming updates return to a regular release cycle.