A week ago, I posted a head-to-head comparison/buyer's guide of the Asus Transformer Pad (TF300), Transformer Prime (TF201), and Transformer Pad Infinity (TF700). The most upvoted comment: how is the internal storage performance? So I sat down to benchmark 6 devices.... and with the help of the team, ended up benchmarking 11:
- HTC One S (S4)
- HTC One X (T3)
- Motorola Droid RAZR Maxx
- Samsung Galaxy Nexus (Android 4.1 - Jelly Bean)
- Samsung Galaxy Nexus (Android 4.0.4 - Ice Cream Sandwich)
- Samsung Galaxy S III (S4)
- Asus Transformer Pad (TF300)
- Asus Transformer Prime (TF201)
- Asus Transformer Pad Infinity (TF700)
- Nexus 7 (Android 4.1 - Jelly Bean)
- Toshiba Excite 7.7
At Mekerz86's suggestion, we used both RL Benchmark and AndroBench, both available for free on the Play Store. Before running any benchmarks, all devices were turned off, turned on, and let sit unused for a few minutes. Although this may have impacted some scores negatively (due to a lack of caching and the like), it was a necessary trade-off to ensure that background apps/processes were limited and fairly consistent; letting the device sit unused after boot up helped ensure all boot-up tasks, syncing, and updates had been completed. Three consecutive trials were run of each benchmark, and those 3 scores were then averaged.
RL Benchmark is an SQLite benchmark that runs thousands of insert, select, update, and delete functions - the same functions Android uses to store data. The resulting number is the total amount of time (in seconds) that it took for your device to run through all the tests. The faster it completed it, the faster your device stores and retrieves data; thus, lower is better.
AndroBench is a little more complicated, and tests Micro, SQLite, and Macro performance. For our tests, we only used the Micro and SQLite scores. Micro measures sequential read/write and random read/write speeds of your device's storage, while SQLite (again) tests insert, update, and delete performance. Sequential speeds are reported in MB/s, and Random speeds are reported in MB/s and input/output operations per second (IOPS). SQLite tests are measured in both transactions per second (TPS) and seconds, though seconds were not used in the test since they directly correlate to TPS and thus overlap.
Not easily graphed.
The resulting output from AndroBench was 12 different numbers per benchmark per device - clearly too many to be individually charted and graphed. The solution: combine the sequential speeds (1) , random speeds (into MB/s, 2, and IOPS, 3), and SQLite scores (4) to come up with 4 different totals. It was the average of these totals I then graphed - much cleaner, don't you think?
Finally, for those who don't know, this is the simplest depiction of the difference between sequential and random access:
Again, this is the total amount of time it took in seconds to execute thousands of SQLite actions; lower is better.
Obviously the winners here are the Galaxy SIII, Galaxy Nexus (ICS), and One S, all about one second apart. With the exception of the Galaxy Nexus (JB), it seems that phones generally lead the pack in SQLite performance. And while this isn't the most scientific test, it appears that Jelly Bean slows things down a bit - the Nexus 7 and Galaxy Nexus (which both run JB) are substantially slower than all the other ICS phones.
Again, these scores are the sum of the various averages.
The Galaxy Nexus absolutely crushed the sequential benchmarks on both ICS and JB, though ICS was more than twice as fast - definitely an outlier. Next up were the One S and One X, followed by the SGSIII.
Random Read/Write: MB/s
Rather surprisingly, the MB/s scores for random read/write speeds were all over the board, ranging from 203.08 (GNex ICS) to 3.86 (Razr). It's a little hard to understand why the range is so massive, since we have 2 "outliers" out of 11 results - and even the best non-outlier (SGSIII, 11.22) is 3 times higher than the lowest score. Let's eliminate the outliers just to get a clearer picture:
All told, the Galaxy Nexus crushes again on both ICS (1st place) and JB (2nd place); the SGSIII checks in at third place. The other devices are all substantially lower.
Random Read/Write: Input/Output Per Second (IOPS)
Once again, it's pretty hard to get a clear picture with the Galaxy Nexus as an outlier. Something is definitely going on with those scores - let's remove them and regraph:
Without the Galaxy Nexus dominating the picture, it's a little easier to see the differences. Excluding the Galaxy Nexus, the SGSIII, One X, and One S once again lead the pack by a wide margin.
SQLite: Transactions Per Second (TPS)
Once again, the Galaxy Nexus dominates so much that it throws off the entire graph. Obviously, it takes first and second place, but so that we can get a clearer comparison of the rest of the scores, let's take it out (... again).
Excluding the Galaxy Nexus, the SGSIII, One X, and One S lead the pack.
There are a few broad conclusions that can be drawn from these tests, general as they may be. First, ICS (Android 4.0) seems to be better about storage speeds than JB (Android 4.1). Second, phones tend to be faster than tablets.
With that said, it's important to keep in mind that these are synthetic benchmarks and are not a direct reflection of real-world performance. The TF700 and Excite 7.7 feel every bit as snappy in real-world use as do the One X and One S. The benchmarks do help show performance differences and can help explain the quality of the experience, such as the likelihood of experiencing "Application Not Responding" (ANR) errors.