30
Jun
2349283746_7ed48c9423_z

If we're being honest, it's hard to deny that one of Android's most obnoxious flaws comes in the category of audio performance. Playing some music is generally fine, but the issues start to become obvious after introducing very high quality audio or trying to achieve precise timing or real-time processing. With the L Developer Preview, it appears that Google is driving to improve upon these weaknesses and give audio performance the shot of adrenaline it needs. In a session at Google I/O 2014 titled "Building great multi-media experiences on Android," Glenn Kasten and Andy Hung took to the stage to explain the improvements appearing in the next revision of Android that can give us access to higher quality sound and greatly reduced lag in audio input.

The advantages for sound quality will come from shifting from 16-bit PCM to floating-point audio. This is about little more than widening the bandwidth for the audio pipeline and reducing the amount of clipping that can occur. According to the duo in the presentation, there should be little or no discernable impact on performance from switching to floating points, and the output will still remain in the form of integers, albeit higher than 16-bit. For the non-audio engineers in the crowd, the only thing you need to know is that Android L will be able to push very high quality audio, and that's going to be great for HD-quality sound. Code samples haven't yet been provided for using floating point audio, but it should be as simple as passing in the newly created ENCODING_PCM_FLOAT flag to the constructor of the AudioTrack class.

2014-06-30_07h45_10

There wasn't much of an explanation for how the team reduced microphone latency, but it was clearly a topic they were eager to demonstrate. Christian House from StarMaker Interactive was brought on stage to explain and demonstrate how his company's app could begin to give real-time feedback to people by using some new low-latency APIs on L. Glenn Kasten acknowledged that there's still a long way to go until the latency issue is truly licked. Unfortunately, it's still not entirely clear which APIs were added or changed, but it sounds like developers may have to stick to the NDK (Native Development Kit) and OpenSL ES to make any headway. Check out this page for more details.

While the improvements to audio are still a work in progress, it's fantastic to see that Android is closing the gap between itself and iOS. Hopefully, this will finally remove the biggest obstacle preventing real-time audio processing, mixers, and professional-grade studio apps from appearing on Android.

Thumb Photo: microphone by Matthew Keefe (CC BY 2.0)

Cody Toombs
Cody is a Software Engineer and Writer with a mildly overwhelming obsession with smartphones and the mobile world. If he’s been pulled away from the computer for any length of time, you might find him talking about cocktails and movies, sometimes resulting in the consumption of both.

  • http://www.CowboyButler.com Cowboy Butler

    Yes - sound quality needs to improve for calls, etc. But on their Multimedia topic, I'll pick on my "favorite" sticking problem: INCREDIBLY bad Bluetooth audio sync in multimedia/video (Netflix, anyone?)

    • http://petercast.net Peterson Silva

      I haven't used Netflix, but for me the problem seems to be the equipment, really, since with the same app and file (a video) I get two different experiences: bt headphones, fine; bt car stereo, terribly out of sync.

    • pallzoltan

      I have a 3-4 seconds delay with bt n my car when listening to music. Not terrible but sometimes unpleasant. Calls are just fine.

    • bobEveryman

      I have the same problem. It's really not much of an issue with audio-only situations (Pandora, listening to podcasts in DoggCatcher, etc.). But add video, or gaming, where the synchronization matters, and BT just suffers horribly.

      I haven't experienced the 3-4 second delay as mentioned, but more like 1/2 to 1 second on my Nexus 5 with LG HBS-700 headphones.

    • mikebel3

      iOS pauses the video for a 1/2 seconds to let the audio catchup and sync. It seems to do it across all apps. Hope Google figures out how to implement that in Android.

  • Alexandre Leites

    Someone else spotted the setting about a new player in development settings?

    • http://www.androidpolice.com/author/cody-toombs/ Cody Toombs

      Yeah, I looked into that a couple of days ago. Here's what I already wrote internally about it:

      Technically, NuPlayer is a counterpart to StageFright, but StageFright isn't much more than a wrapper around AwesomePlayer, so the two are close enough to being synonymous. Both are very low-level in the framework.

      NuPlayer isn't actually new, it's been around since Honeycomb. It was introduced to handle Apple's HTTP Live Streaming standard. Since then, it gained a few other tricks, but it looks like it's a streaming-centric player (unless that has changed in the latest builds). AwesomePlayer handles local playback and some of the streaming stuff NuPlayer doesn't implement. In cases where the two cover the same types of codecs/sources, AwesomePlayer is the default.

      Since there are no documented details, we don't have source code for L, and the explanation simply says "instead of," I can only assume this setting just changes the default to sway towards NuPlayer instead of AwesomePlayer for the stuff that both of them support. I imagine this toggle will only affect those apps that use the higher-level classes instead of calling on these directly.

      Without any information about what has really changed, I don't think there's enough to write about. It would be just be an explanation of 3+ year old libraries and a couple of guesses about what might have changed. I could write an app that could poke around and test a few things, but that would consume WAY more time than it's worth to just produce slightly more educated guesses.

      At least for now, there wasn't much of a point in writing up a post about it. I'm pretty sure this really is just for developers of media players to test NuPlayer. Perhaps it will become the default in the future, sorta like how ART was trialed and ultimately replaces Dalvik. Whatever the case, there's just not enough information to actually cover it.

      • Matthew Gardner

        As a side note having this developer option enabled froze my camera when I tried to take a picture. Tried it with the dev option on and off, Nuplayer preferred consistently froze my N5 camera when the shutter button was hit, whereas Awesomeplayer preferred did not. It's in the issue tracker as well https://code.google.com/p/android-developer-preview/issues/detail?id=49 .

        • Will Tisdale

          Playback of the camera shutter sound...

          Player hangs, camera service hangs and stops talking to the Camera app. Camera app then hangs. Whole thing generally takes a reboot to fix as the camera service doesn't like to recover.

          I remember having similar issues when doing ROM development years ago and hacking around some of the weird audio quirks on Android 2.3.x.

    • mgamerz

      Ahh, the classic volume bar ruining screenshots. Then the follow up screenshot that says 'saving screenshot' in the notification bar. Followed by the used screenshot, for a total of three.

      • Crispin Swickard

        After the first volume bar fail I will at least wait for it for write out, and clean the notification before the next attempt. I try to keep it classy. :)

      • Klappa

        I don't get it?

        • mgamerz

          Its the three stage process to getting a good sctreenshot.

  • http://www.failuretodream.com/ lostboardmayhem

    I hope they add surround sound output too by the time Android TV comes out or it is a no go for my home media

    • Walkop

      That's already covered. My Nexus 10 outputs surround sound, IIRC.

      • http://www.failuretodream.com/ lostboardmayhem

        Glad to hear that, hopefully there will be set top boxes with optical/digital coax then

  • Nabil L

    Guys I've been looking at the Google Design site for the past few days, and here's what I came across today. Those very (too much) simplistic navigation bar buttons are probably not gonna make it to the final version (I hope, at least), and these here might replace them. It looks MUCH better.

    Here's the link to it : http://www.google.com/design/spec/components/progress-activity.html#progress-activity-types-of-indicators

    • rmkilc

      Don't hold your breath.

    • matt

      But....but..I like my PlayStation buttons!

      • Crispin Swickard

        I was thinking it would be hilarious if Sony phones (even though they follow the regular 3 option standard) would implement a 4th X button. :D

    • Imparus

      When talking about spotting stuff in the guidelines document, we might also get the same implementation of permission as iOS

  • kekkojoker90

    Real time audio isn't near even with this release. A tecnichan said we can have with this release about 20ms of latency but for a good performance is needed maximum 9/10 ms of latency.

    • mikebel3

      That's disappointing. Looks like ios is still better suited for music creation.

      • kekkojoker90

        sadly yup. But he also says is a good improvement. Ios have an huge advantage because apple ported thei coreaudio driver to ios from osx. This driver are developed in many year so it's only matter of time for android to catch

      • Jacob Martin

        IOS is also better for music listening both from the user point of view and otherwise. Have you ever gotten a text message whilst listening to music on Android?

        Quality is also different from version to version, and obviously device to device.

        • hoosiercub88

          On my Note II, music volume would be lowered and the notification would sound.

          On my Moto X, it temporarily mutes the audio, sounds the notification, then automatically resumes.. As with sound quality, it's again of case to case basis, as different OEMs handle small details like that differently.

          • Jacob Martin

            The Android devices I have and use mute the music and then resume. Xbox Music and some others ignore notification or lower the volume, but most mute or stop the music. Which is annoying if you get or are trying to have a text conversation.

            All other mobile platforms handle it much better.

          • Milind

            0n every Android device I have used Galaxy S3, LG G2 and HTC One M8, the volume lowers, beeps with the notification and resumes. The iPhone does the same on Google Play Music. I didn't use the default music app. On the GS3, I *think* (it's been a while), I used to have a setting where I turned off notification volume. That was the default in my day to day commute, but I went back to the normal behavior on long road trips. What behavior would you like?

    • Adrian Meredith

      considering 60fps is a latency of 16 ms anything more is just overkill for the huge majority of use cases. Its not like android is going to be used professionally anytime soon.

      • shonangreg

        Just download one of the musical instrument apps, especially a drum machine, and try to say that again with a straight face. Tap... wait... wait... bam! Tap... wait... wait... bam! Getting very low latency is going to open our android devices up a lot for music creation.

        Funny none of these articles are mentioning the instrument side of things. Maybe all musicians are just using iOS already and have given up on android.

        • mikebel3

          That's what I think. I love my Android phone but tap and play is just impossible. You can get around it somewhat with apps like Caustic that let you edit and move midi notes to their correct locations. But it's still not as easy as on an idevice + quantize.

      • mikebel3

        I use an iPad and the MusicStudio app to mess with ideas. Ya, it's not a full fledge DAW but it would be nice to have a choice between Android or iOS.

        • Milind

          I can see that. I had a Gmail account for years that I never used. I bought the G1, saw how contacts got synced to the cloud and I could edit contacts on the Web with a much better keyboard and started using Gmail, then Google Voice, Google Music and now am totally in the Google ecosystem. But it started with Contacts!

    • http://mwinter.in/ Yan Gabriel Minário

      What about a Galaxy Nexus running Android L? I think it's the phone with the least audio delay.

    • Mike Reid

      Input latency long sucked anyway, like over 100 ms.

      Problem here is that we often end up stuck with whatever audio HALs the OEMs come up with. Samsungs seem to be much better than HTCs.

      Another reason to go Nexus, not even GPE, which often (almost always ?) uses the OEMs HALs, as seen on HTC One and more.

      Google can only guarantee how well Nexus does, with fully functional open source HALs.

      DO Note that OEM HALs are usually not opened sourced. Only the Linux Kernel portion forces GPL release.

      I'd like to see Google try to enforce sound performance more in their Compatibility Test Suite.

  • AfrodanJ

    That presentation was downright offensively bad

    • niuguy

      Yeah, but it was bad enough that it was, to use my wife's words, cute. haha

  • Bruno

    They should team up with AVID to develop the best audio phone.

  • http://www.androidpolice.com/ David Ruddock

    Floating point processing is relevant to music creators, and even then, only at the very technical end of the music creation business. I don't know that there could be a less relevant change to audio made in the OS. I'm not even sure why they're making a fuss about it - I'm guessing this will affect precisely zero people, because no one uses Android as a serious music or audio production platform.

    In fact, FP audio has generally seen a backlash in recent years because it offers basically zero perceptible benefit and potentially can be destructive in audio processing: http://www.tvtechnology.com/audio-etc./0193/fixed-point-vs-floating-point-numbers-in-audio-processing/266375

    It can be beneficial in an editing environment, but only in a few, limited circumstances.

    • http://www.androidpolice.com/author/cody-toombs/ Cody Toombs

      It's worth pointing out that Floating point processing is an optional flag. Developers can still pick 8-bit and 16-bit PCM if they like (or make it an option for users, depending on the app).

      I believe the whole point of both of these improvements (and others mentioned in the video) is to make Android suitable for serious music and audio production, so people will eventually have a choice other than iOS.

    • Nikhil

      There are many apps that can be used as percussion, drums, guitars, etc. by a vocalist while performing live, instead of using a live band / orchestra. Android has a few of such apps, but the professional grade ones are only on iOS. I personally had to buy an iOS device only for one of the music app - the developer has strongly declined making one for Android due to latency and piracy issues.

      I don't know if this particular change (PCM to FP) will improve the performance, but if they bring latency down to iOS levels, it is going to be beneficial to a LOT of users, and not zero!

  • NinoBr0wn

    At this point, if Android L doesn't reveal to me the truth to life and all it's splendors, I will be very disappointed, and I'll get an iPhone 6 instead.

    OK that last bit is a blatant lie, but I can't wait for L.

  • GraveUypo

    so what about the screen latency. they doing anything about that yet? that bothers me a hundred times more than audio latency.

    • Itchy_Robot

      They supposedly fixed it with "project butter". But obviously that was not the case. As a musician I have given up on Android for music instrument apps. It seems Google has decided to give that sector to iOS. Sucks for me since I am already invested in Android devices.

    • Dominic Powell

      See my post above.

  • http://www.mcgeeb.com Bryan McGee

    Here's hoping that an iRig port could finally happen.

  • nad

    really sad, IOS has 5 ms latency, and android hardly does 20 ms, in addition to touch latency

    • Dominic Powell

      Android's touch latency has gotten much much better, to the point where android smartphones are ahead of the iphone 5s.

      the HTC one M8, Moto G, LG G3 and even the Moto E have better touch latency (48ms - 62ms) than the iphone 5s which has the same touch latency as the Nexus 5 ( ~75 ms) (the ipad is 52 - 55 ms). And this is without the touch latency improvements that were mentioned at I/O in the L release

      • Milind

        I actually disliked HTC One M8's touch latency because it was too sensitive! They reduced the latency, but didn't control it. I was inadvertently moving icons, starting apps etc, just handling the phone. I hope Google doesn't make the same mistake. I only saw one other person complain about it online, but it used to drive me nuts. I returned the HTC One for other reasons. Now I'm waiting for the G3.

        • Dominic Powell

          Well handle a nexus 5 that has the same

          • AndyRoberts

            Never had a problem with latency really in any of the Android phones in the last couple of years. Except the One M8.

    • 32BitFloater

      For music creation, tap-to-audio is important and iOS is delivering 30-50 ms depending on app and device. There is NO WAY iOS is delivering 5 ms.

  • blindexecutioner

    I assume this should help tremendously with emulators right?

  • mjku

    It is slightly embarrassing it has taken the Android team this long to even get this far. That said, I'm grateful to their efforts to improve the situation and I'm excited for further improvements. Keep it up, guys!

    • hoosiercub88

      Mind you, iOS had the audio drivers essentially ported over from OSX.. They had it well before, Google started from scratch with Android.

  • ssdickey

    No wonder Android audio latency is terrible... Just listen to these guys... Its all a big joke to them. Google really needs to get these guys off the project and get another perspective on this issue.

    • Mauro

      Being awful at talking to a room full of people doesn't mean they don't know their stuff.

  • bse88

    If anyone in Google have tried to record audio with a Nexus 4, they know they own us, the users, a big apology in the shape of a "L".

  • matt

    They should be working on the audio levels of for the n5. The new kit kat updates seem to have broken the audio boosters for me as I hear no difference In the loudness

  • someone755

    "Hey, Christian, I can hear you"
    I wish I didn't tho xD

  • mariano zorrilla

    Are does the WORST acting performance EVER?! Please god, kill me!