21
May
nexusae0_cyanogen-thumb_thumb

Last year, Samsung got into some hot water for including an automatic "high power mode" for certain apps, dialing up the processor and GPU scaling. There's nothing wrong with that in theory, but these changes were enabled specifically for benchmark apps, giving the benchmarks results that, while not technically incorrect, were artificially inflated and unlikely to be indicative of everyday performance.

Yesterday, popular custom ROM family CyanogenMod added similar state-dependent modes to its latest batches of code, and as soon as the power profiling function was added, specific triggers for the popular Quadrant and Antutu benchmarks were added as well. This means that once the changes make it into nightly builds, devices running the latest CyanogenMod builds will see higher scores in Quadrant and Antutu. Again, while these higher scores aren't technically incorrect (the hardware is operating within its standard parameters), they do reflect a level of performance that's unlikely to be achieved in other apps, like high-powered games, without similar profiling. User-accessible controls for specific apps are marked for development later.

Performance Profiles is going to grow into a much more powerful feature which can apply advanced optimizations or power saving techniques depending on both the state of the hardware as well as the current applications in use. -Steve Kondik, CyanogenMod code review

Cyanogen, Inc. founder and CTO Steve Kondik merged the changes into the primary CM11 build himself. When a user commented that Futuremark was likely to ban devices running CyanogenMod from the public rankings of the 3DMark benchmark if a similar mode was enabled for that app, Kondik had this to say:

I think its totally fair. Every Qualcomm device already does this, and you can already engage performance mode manually. This will improve the situation because you'll be able to turn it on and off as you please.

Also, it won't really do much for 3dmark since our GPUs already scale pretty damn well.

Kondik went on to say that HTC's implementation of similar power profiles on the new One M8 was one of his reasons for making the change. Notably, Futuremark has indeed banned the One M8 from its public rankings for the above behavior.

Again, the mere function of changing the power profile to increase or decrease speed based on the needs of the current app load isn't in question - that's simply good software design. It's the tagging of benchmark apps, which are ostensibly designed to be objective tests of typical hardware power, which will cause some users to balk.

<!-- Automatic power profile management per app.

Each item should list the fully-qualified activity

name and the power profile id, separated by a comma. -->

<string-array name="config_auto_perf_activities" translatable="false">

<item>com.aurorasoftworks.quadrant.*BenchmarkExecutionActivity,2</item>

<item>com.antutu.ABenchMark.*,2</item>

<item>.*com.antutu.benchmark.*,2</item>

</string-array>

A move like this would probably go unnoticed for most custom ROMs. But CyanogenMod is installed on millions of devices and, more importantly, now has a for-profit company developing and promoting it. That causes a problem: many users install custom ROMs to gain performance benefits over stock manufacturer software, but with these changes made, the Quadrant and Antutu benchmarks are not objectively reliable tests of comparable performance between the two ROMs. The change could be seen as a way of making CyanogenMod look artificially better in comparison to stock software.

Source: CyanogenMod code review 1, 2, 3 - thanks, Utkarsh Gupta!

Michael Crider
Michael is a native Texan and a former graphic designer. He's been covering technology in general and Android in particular since 2011. His interests include folk music, football, science fiction, and salsa verde, in no particular order.

  • Paarth Desai

    giving me more reason to hate cm drastically

    • http://www.geordienorman.com/ George Byers

      Ugh... I hope your not one of them Foss people and the reason why you hate cyanogenmod is because they are a commercial company now..... Wake up and welcome to America.... You and they are both entitled to make money, it's called capitalism....

      • ddpacino

        Is Cyanogenmod, Inc. even a US company..?

        • http://www.standupforkids.org blackroseMD1

          Based in Seattle, so yeah.

    • nawa

      DAE THINK CM IS LITERALLY HITLER AND A CORPORATE SHILL LOL

      (sorry)

  • Dani R.

    Cyanogenmod has turned into an awful ROM. Not only the is ROM slower than other ROMs, they're also heading in the wrong direction...

    • someone755

      It's funny.
      'Cause it's true.

      Tho really it's sad and I am disappointed.

  • Nate

    You guys need to stop bitching about cm every other company arleady does it and at least cm is doing it for a reason, hence the picking and choosing of apps to come in the future obviously the first apps ur gonna add to this feature are benchmark apps to make sure your code is working :/

    • dentou

      for a group of modders that always critize that practice and now they are doing it. Does not talk well about their credibility.

      • ml

        Please point me to the sources, where they criticize this. Otherwise, I'll question your credibility ;)

  • derk p

    cyanogenmod does not work well with pebble watch. i had to roll back
    to stock and my pebble is flawless, on CM it will not stay connected,
    notifications dont always come through, and if i walk out of range and
    come back, it doesnt reconnect. pretty sad as ive loved CM but xposed
    has made them irrelevant to me.

    • silver_arrow

      My pebble works fine with my Nexus 4 on CM. Yeah sometimes i have to manually connect it but that's no big deal.

      • Matt Bernstein

        +1 my pebble works fine on my OnePlus running 11s

        (no i dont have invites)

        • Mainul Mizan

          How is the daily life experience of using 1+1?

          • Matt Bernstein

            I love it! Get device

  • http://petercast.net Peterson Silva

    Is it bad that I don't really care?

    I could not give two fucks about benchmarks. The phone has to feel snappy in everyday usage; not stuttering, freezing, etc. I understand that the better the score less likely the phone is perform badly considering such criteria, but nowadays most mid-to-high-end phones can do that just as well - it's like the Moto E vs S5 thing.

    Perhaps I just don't use my cell phone for intensive tasks that would require such power.

  • mlj11

    So it's like an in-built version of the Per App Modes app?

  • Kurt Frost

    People still look at benchmarks?

    • Krzysztof Jozwik

      I always preferred real world usage to see what seems smoother.

  • Matthew Fry

    Just let everybody do it and then all the sudden the benchmarks mean something again.

  • Brin

    CyanogenMod developer here (I don't work for Cyngn Inc or speak on behalf of the whole team) :
    Remember to keep in mind that the implementation of this patch depends on support from the device specific power HAL setup.
    If you're device doesn't implement the performance profiles, then nothing will change and this change won't matter.
    For most devices the profiles support is still missing, so don't expect better benchmark scores just right now (only a few that are actually enabled right now IIRC).
    It will be up to the maintainers to support that by enabling it in their device trees for supported devices..

    • https://profiles.google.com/u/0/sagarsiddhapura/about SAM

      Its not about device supports performance boosting or not. Its about presenting true facts and figures that represent the device for all applications. If you boost scores just for benchmarking apps, people will be deceived by considering this scores. Benchmarking is a good measure on how your device can perform and rigging this is clearly cheating. Show users what the device is capable of by presenting true scores applicable for all apps. Negative vote for this move!

  • https://profiles.google.com/u/0/sagarsiddhapura/about SAM

    Device should present true facts and figures that represents it for all applications. If you boost scores just for benchmarking apps, people will be deceived by considering this scores. Benchmarking is a good measure on how your device can perform and rigging this is clearly cheating. Show users what the device is capable of by presenting true scores applicable for all apps. Negative vote for this move!

  • Anonymous

    I think cyanogenmod is doing the right thing. Since Benchmark is measuring the devices "MAX" performances rather than devices normal/real world performances.

  • aenews

    Honestly, this isn't a big deal. This feature simply deals with scaling the clock speed in different apps first of which will be benchmarks. It doesn't magically raise the clock speed beyond default. The problem with Samsung's feature was that clock speeds would reach higher than normal speeds not available to other apps on stock kernel. For anyone who uses Performance Governor or raises scaling settings on other governors, this doesn't even make a difference. Besides, the governor almost always scales to the maximum clock speed on both the CPU/GPU when running intensive tasks like benchmarks.