21
May
nexusae0_cyanogen-thumb_thumb

Last year, Samsung got into some hot water for including an automatic "high power mode" for certain apps, dialing up the processor and GPU scaling. There's nothing wrong with that in theory, but these changes were enabled specifically for benchmark apps, giving the benchmarks results that, while not technically incorrect, were artificially inflated and unlikely to be indicative of everyday performance.

Yesterday, popular custom ROM family CyanogenMod added similar state-dependent modes to its latest batches of code, and as soon as the power profiling function was added, specific triggers for the popular Quadrant and Antutu benchmarks were added as well. This means that once the changes make it into nightly builds, devices running the latest CyanogenMod builds will see higher scores in Quadrant and Antutu. Again, while these higher scores aren't technically incorrect (the hardware is operating within its standard parameters), they do reflect a level of performance that's unlikely to be achieved in other apps, like high-powered games, without similar profiling. User-accessible controls for specific apps are marked for development later.

Performance Profiles is going to grow into a much more powerful feature which can apply advanced optimizations or power saving techniques depending on both the state of the hardware as well as the current applications in use. -Steve Kondik, CyanogenMod code review

Cyanogen, Inc. founder and CTO Steve Kondik merged the changes into the primary CM11 build himself. When a user commented that Futuremark was likely to ban devices running CyanogenMod from the public rankings of the 3DMark benchmark if a similar mode was enabled for that app, Kondik had this to say:

I think its totally fair. Every Qualcomm device already does this, and you can already engage performance mode manually. This will improve the situation because you'll be able to turn it on and off as you please.

Also, it won't really do much for 3dmark since our GPUs already scale pretty damn well.

Kondik went on to say that HTC's implementation of similar power profiles on the new One M8 was one of his reasons for making the change. Notably, Futuremark has indeed banned the One M8 from its public rankings for the above behavior.

Again, the mere function of changing the power profile to increase or decrease speed based on the needs of the current app load isn't in question - that's simply good software design. It's the tagging of benchmark apps, which are ostensibly designed to be objective tests of typical hardware power, which will cause some users to balk.

<!-- Automatic power profile management per app.

Each item should list the fully-qualified activity

name and the power profile id, separated by a comma. -->

<string-array name="config_auto_perf_activities" translatable="false">

<item>com.aurorasoftworks.quadrant.*BenchmarkExecutionActivity,2</item>

<item>com.antutu.ABenchMark.*,2</item>

<item>.*com.antutu.benchmark.*,2</item>

</string-array>

A move like this would probably go unnoticed for most custom ROMs. But CyanogenMod is installed on millions of devices and, more importantly, now has a for-profit company developing and promoting it. That causes a problem: many users install custom ROMs to gain performance benefits over stock manufacturer software, but with these changes made, the Quadrant and Antutu benchmarks are not objectively reliable tests of comparable performance between the two ROMs. The change could be seen as a way of making CyanogenMod look artificially better in comparison to stock software.

Source: CyanogenMod code review 1, 2, 3 - thanks, Utkarsh Gupta!

Michael Crider
Michael is a native Texan and a former graphic designer. He's been covering technology in general and Android in particular since 2011. His interests include folk music, football, science fiction, and salsa verde, in no particular order.

  • Paarth Desai

    giving me more reason to hate cm drastically

    • http://www.geordienorman.com/ George Byers

      Ugh... I hope your not one of them Foss people and the reason why you hate cyanogenmod is because they are a commercial company now..... Wake up and welcome to America.... You and they are both entitled to make money, it's called capitalism....

      • ddpacino

        Is Cyanogenmod, Inc. even a US company..?

        • http://www.standupforkids.org blackroseMD1

          Based in Seattle, so yeah.

    • nawa

      DAE THINK CM IS LITERALLY HITLER AND A CORPORATE SHILL LOL

      (sorry)

    • b3nt

      See my response to SAM. I hope you do not take offense to my example, that is not my intent.

  • Dani R.

    Cyanogenmod has turned into an awful ROM. Not only the is ROM slower than other ROMs, they're also heading in the wrong direction...

    • someone755

      It's funny.
      'Cause it's true.

      Tho really it's sad and I am disappointed.

  • Nate

    You guys need to stop bitching about cm every other company arleady does it and at least cm is doing it for a reason, hence the picking and choosing of apps to come in the future obviously the first apps ur gonna add to this feature are benchmark apps to make sure your code is working :/

    • dentou

      for a group of modders that always critize that practice and now they are doing it. Does not talk well about their credibility.

      • ml

        Please point me to the sources, where they criticize this. Otherwise, I'll question your credibility ;)

      • b3nt

        Actually, I believe this was actually a lot of noise created by tech pundits and not necessarily actual techs. Not everyone knew the facts at that time, so they colored the situation the way that suited them ... to get their articles read.b3nt

  • derk p

    cyanogenmod does not work well with pebble watch. i had to roll back
    to stock and my pebble is flawless, on CM it will not stay connected,
    notifications dont always come through, and if i walk out of range and
    come back, it doesnt reconnect. pretty sad as ive loved CM but xposed
    has made them irrelevant to me.

    • silver_arrow

      My pebble works fine with my Nexus 4 on CM. Yeah sometimes i have to manually connect it but that's no big deal.

      • Matt Bernstein

        +1 my pebble works fine on my OnePlus running 11s

        (no i dont have invites)

        • Mainul Mizan

          How is the daily life experience of using 1+1?

          • Matt Bernstein

            I love it! Get device

  • http://petercast.net Peterson Silva

    Is it bad that I don't really care?

    I could not give two fucks about benchmarks. The phone has to feel snappy in everyday usage; not stuttering, freezing, etc. I understand that the better the score less likely the phone is perform badly considering such criteria, but nowadays most mid-to-high-end phones can do that just as well - it's like the Moto E vs S5 thing.

    Perhaps I just don't use my cell phone for intensive tasks that would require such power.

  • mlj11

    So it's like an in-built version of the Per App Modes app?

  • Kurt Frost

    People still look at benchmarks?

    • Krzysztof Jozwik

      I always preferred real world usage to see what seems smoother.

  • Matthew Fry

    Just let everybody do it and then all the sudden the benchmarks mean something again.

  • Brin

    CyanogenMod developer here (I don't work for Cyngn Inc or speak on behalf of the whole team) :
    Remember to keep in mind that the implementation of this patch depends on support from the device specific power HAL setup.
    If you're device doesn't implement the performance profiles, then nothing will change and this change won't matter.
    For most devices the profiles support is still missing, so don't expect better benchmark scores just right now (only a few that are actually enabled right now IIRC).
    It will be up to the maintainers to support that by enabling it in their device trees for supported devices..

    • https://profiles.google.com/u/0/sagarsiddhapura/about SAM

      Its not about device supports performance boosting or not. Its about presenting true facts and figures that represent the device for all applications. If you boost scores just for benchmarking apps, people will be deceived by considering this scores. Benchmarking is a good measure on how your device can perform and rigging this is clearly cheating. Show users what the device is capable of by presenting true scores applicable for all apps. Negative vote for this move!

  • https://profiles.google.com/u/0/sagarsiddhapura/about SAM

    Device should present true facts and figures that represents it for all applications. If you boost scores just for benchmarking apps, people will be deceived by considering this scores. Benchmarking is a good measure on how your device can perform and rigging this is clearly cheating. Show users what the device is capable of by presenting true scores applicable for all apps. Negative vote for this move!

    • b3nt

      I know this is an old(ish) thread, but:

      It will be the true score available for all apps... they just have to be added to the performance profile first! We all know that this profile list will either allow you to choose what is in it or at least to be hacked at some point in the future. Additionally, you may have a fundamental misunderstanding of what is going on here, so let me educate you :)

      Keep in mind the true purpose of the benchmarks are to show the max potential of a device. Benchmark sites do not want to list it because it does indeed threaten their very existence... with a profile-based CPU profiling system, CPU benchmarks are neither as relevant as they once were, nor do they now adequately reflect performance of all applications **by default**. This muddies the water a good bit, because there is a much larger argument to be had for what to do when overclocking enters into the fray, etc.

      Think of it this way: Like the "caste" (class) system in India that, like it or not, still exist. While not so good for the lower castes, no one can argue that the higher castes benefit easier access to improvement of their lives such as good jobs or housing, etc (think of it as better access to resources). With people, this is a terrible human rights violation, as no human is more deserving of food/water/air (well except for maybe perpetrators of horrific crimes)... but with apps, it is just a smart way to save battery by limiting access to resources for apps with lower resource requirements (email, twitter, etc), and giving more resources to those that need performance (games, camera etc).

      I would say that a benchmark application uses a huge amount of resources, doesn't it? Don't games? Don't graphics or video editing apps? Why should they be prompted to more resources and not the benchmark app? When you think about it that way, you will realize your stance is incorrect.

  • Anonymous

    I think cyanogenmod is doing the right thing. Since Benchmark is measuring the devices "MAX" performances rather than devices normal/real world performances.

  • aenews

    Honestly, this isn't a big deal. This feature simply deals with scaling the clock speed in different apps first of which will be benchmarks. It doesn't magically raise the clock speed beyond default. The problem with Samsung's feature was that clock speeds would reach higher than normal speeds not available to other apps on stock kernel. For anyone who uses Performance Governor or raises scaling settings on other governors, this doesn't even make a difference. Besides, the governor almost always scales to the maximum clock speed on both the CPU/GPU when running intensive tasks like benchmarks.

Quantcast