3DMark came out a while back to give your Android device's GPU a rating, and now Futuremark's other benchmarking tool has arrived in the Play Store. PCMark will analyze the overall performance of your phone or tablet, rather than focusing on individual components. At the end you get a number. Is that number useful? Maybe.
Samsung was the first to selectively boost system performance when a benchmark app was run, but it was forced to backpedal pretty quickly on that one. The latest OEM to try and sneak one past the benchmarks is Huawei with its new-ish Ascend P7. Futuremark is wise to this game, though, and has pulled the P7 from the 3DMark top phone charts.
Last year, Samsung got into some hot water for including an automatic "high power mode" for certain apps, dialing up the processor and GPU scaling. There's nothing wrong with that in theory, but these changes were enabled specifically for benchmark apps, giving the benchmarks results that, while not technically incorrect, were artificially inflated and unlikely to be indicative of everyday performance.
Yesterday, popular custom ROM family CyanogenMod added similar state-dependent modes to its latest batches of code, and as soon as the power profiling function was added, specific triggers for the popular Quadrant and Antutu benchmarks were added as well.
Shortly after the new Android Runtime made its grand entrance, I ran a pretty exhaustive (and exhausting) series of performance benchmarks that showed ART wasn't really ready to blow us away. At the time, I opted to avoid the topic of battery life because it is so difficult to test accurately and with unbiased, meaningful results. As it turns out, that was dumb. Yup, so many of you have asked, I finally had no choice but to dive in and run a battery of tests on...well, the battery.
One of the many vectors which delivers leaked information on upcoming phones is the device benchmark score. Many apps upload the results to online leaderboards for all to see. There's quite a bit of information to be gleaned from these online listings – basically the entire spec list. Now Sony is looking to plug this leak by blocking benchmarking apps.
By now you've probably heard about ART and how it will improve the speed and performance of Android, but how does it actually perform today? The new Android Runtime promises to cut out a substantial amount of overhead by losing the baggage imposed by Dalvik, which sounds great, but it's still far from mature and hasn't been seriously optimized yet. I took to running a battery of benchmarks against it to find out if the new runtime could really deliver on these high expectations.
AnTuTu is one of the premiere full system benchmarks available on Android devices. The developers claim to have over 100 million users, and they're all getting a big update to version 4.0. Not only does the new AnTuTu look better, but the way it determines scores has been tweaked to better support faster devices.
Hardware enthusiasts are probably already aware of Futuremark and its PCMark software, a standard for testing and comparing computer hardware for years. PCMark is popular among reviewers and users for its comparison of hardware on standards that are more likely to reflect real-world, typical usage. Today Futuremark announced that it's bringing the software to the "Big three" mobile operating systems, Android, iOS, and Windows Phone. Futuremark's press release did not include a date.
Do you enjoy knowing how fast things are? Then Qualcomm's revamped Vellamo suite is probably something you should check out. Vellamo has been the web benchmark of choice on Android for some time now, but this new update brings some major improvements.
First, the UI: it's completely different. It's actually really nice, certainly the prettiest benchmarking tool I've ever seen on Android (I mean, who really cares, but still). Just look at the screenshots:
The next big change comes in the form of a brand-new CPU benchmark called Metal.