Last week, we attended an event at Qualcomm's corporate headquarters in San Diego to test out the company's latest high-end smartphone chip, the Snapdragon 845. We ran some benchmarks and ate some food. There was probably more eating than benchmarking, if I'm honest.

Full disclosure: Qualcomm paid to fly me out to San Diego, put me up in a nice little hotel for three nights next to their campus, and fed me lots of food and bought me lots of drinks. Of particular note: some pretty delicious hot wings at midnight.

Introduction

First and foremost, let me be clear that we place essentially zero weight on benchmark results as a reflection of real-world performance or user experience. Benchmarks are benchmarks, they're numbers. We publish this information for strictly informational purposes. No one here is saying one phone is "better" than another because it benchmarks higher. With that out of the way...

Featuring a new CPU design that's up to 25% faster and a GPU with equally impressive improvements (up to 30% faster), it's safe to say that expectations for the 845 have already been pretty well set. Our early benchmarks of Qualcomm's Snapdragon 845 reference platform - the most phone-like reference phone I think Qualcomm's ever made - confirm that the new chip is, indeed, faster than the old chip. How much? Numbers follow, but let's talk about the nuts and bolts a bit before we get started.

Hard to believe something so small can do so much - and yes, that's the full Snapdragon 845 SoC.

Like the 835 last year, Qualcomm tells us that basically everything on the 845 is new. Though the new CPU reference design (based on ARM's A75 and A55 cores) and a beefed-up GPU are the big performance stories, as they tend to be every year, Qualcomm's also introduced new iterations of hardware that you can't really benchmark quite so easily. The Hexagon DSP, for example, is two to three times more powerful than the previous generation for AI tasks, and we're really starting to rely on our smartphone's DSPs for accomplishing the increasingly complex heterogeneous workloads that things like on-device machine learning require. (Qualcomm is aware that machine learning and AI performance benchmarks do exist, but they're in a pretty early state.)

The image signal processors, used by the cameras, are new, and Qualcomm continues to enhance the security of its chips with each passing generation. Support for HDR video encoding means that, as far as we know, Qualcomm Snapdragon 845 devices will be the first smartphones capable of capturing HDR video. The new X20 LTE modem supports true dual-SIM LTE, faster peak speeds through five-band carrier aggregation, and various other improvements.

Most of what we're testing today, though, relies rather strictly on the CPU and GPU, none of these other elements (you can throw in memory, too). And that brings us to the testing.

The testing and the phones

Qualcomm's Snapdragon 845 reference phone is very nearly a normal-looking smartphone. I say "very nearly" because it's got little stickers on it to cover up various engineering codes and security watermarks, and some parts of it just look rather unrefined and otherwise unfinished. Of course, this is an engineering test platform - it doesn't need to look pretty. And it mostly doesn't. But compared even to last year's Snapdragon 835 reference platform, the new 845 reference device is definitely thinner and sleeker. The idea that it's essentially a good stand-in for a real phone isn't something you'd really scoff at - there's no fancy cooling or anything, and overall it seems like a pretty off-the-shelf solution. You could use this thing as a normal phone, aside from the fact that it's running a ton of debug tools and probably isn't especially stable.

A couple of technical considerations: it is running Android 8.0 and features 6GB of LPDDR4X RAM. And, in case you were wondering: No, the Snapdragon 845 reference platform is not waterproof (somebody asked this).

Testing was done using the same kind of benchmark apps you'd find on the Play Store, minus the fact that Qualcomm's versions are all modified (these modified APKs are provided by the benchmark developer) not to connect to the internet. Qualcomm also has conducted its own testing of these benchmarks internally, and provided us some score ranges as a reference in the event we encountered any abnormal results. During my own testing, I saw no results that deviated significantly outside the expected range.

Other phones I tested after the fact at home. Those devices included an international version of the Huawei Mate 10 Pro (Android 8.0), a US Galaxy Note 8 (Android 7.1), Galaxy S8+ (Android 7.0), Pixel 2 XL (Android 8.1), and OnePlus 5T (Android 8.0). All were freshly rebooted. With that, let's get to the results.

The results

The raw results are here in nearly-unreadable table form for you to peruse. I'd suggest moving on to the visualizations, though, and referring back to these after the fact.

Geekbench

First up is the well-known Geekbench suite, version 4. This test is a synthetic benchmark designed to measure raw CPU performance.

GFXBench GL Benchmark

GFXBench contains a variety of Open GL ES 3.1, 3.0, and 2.0 graphics benchmarks. The tests are listed in descending order based upon their complexity, starting with the newest benchmark which tests advanced GL ES 3.1 features. These are all "offscreen" tests, meaning all were rendered at the same resolution regardless of the device's actual, physical resolution.

AnTuTu

AnTuTu is a holistic benchmark that combines things like storage speed, RAM, GPU, CPU, and other performance into a single score. While it's not the most helpful in this sense (after all, who knows what kind of storage Qualcomm's reference platform utilizes), it's a very commonly-used benchmark and at least worth throwing up here.

3DMark

3DMark is an immensely popular graphics benchmark utility on desktops, and the mobile version has picked up steam over the years as well. Our test is the new OpenGL ES3.0/3.1 Slingshot Unlimited, which uses the same resolution to render the test across all devices at the same resolution.

Web benchmarks: Octane, Kraken, SunSpider, JetStream

Web benchmarks like Google Octane and SunSpider measure Javascript performance. Octane, I've found, can be a pretty reliable indicator of a given platform's browser performance, but I'd hardly call it an end-all-be-all. JetStream is the replacement for SunSpider, which has been deprecated.

For Kraken and SunSpider, lower is better. Octane and JetStream are higher is better benchmarks.

Comparing across chipsets

Finally, I've run some of the above benchmarks (the most demanding ones) on four different Qualcomm chipsets of various tiers and generations. Devices used in this comparison feature the Snapdragon 626, 821, 835, and the 845 (on the reference platform). The devices used, for completeness, are the Moto Z2 Play, ZTE Axon M, OnePlus 5T, and 845 reference platform. This comparison is just designed to give you a sense of how a mid-range chip, an older flagship chip, and the best Qualcomm chipset currently available compare to the upcoming 845 platform.

Benchmarks included here are Geekbench, 3DMark Slingshot (3.1 and 3.0 unlimited), GFX Bench (Car and Manhattan 3.1 1080 offscreen), and the Jetstream web benchmark.

Conclusion

Unsurprisingly, Qualcomm's new chipset performs better in benchmarks than its previous one - consistently. Graphics performance seems very substantially improved, and CPU performance overall seems like a good bump over the 835. Benchmarks aren't completely meaningless - they do serve a basic purpose in evaluating a given set of hardware's capabilities within a limited set of tasks.

Of course, these tests don't take into account how much more efficient the Snapdragon 845 is than the 835, or a host of new features it contains. They merely illustrate that the GPU and CPU are faster than the GPU and CPU of the last chip. And that's not surprising.

This, of course, can translate into an end user experience benefit. But devices like the Essential Phone show that, even with top of the line hardware, the best user experience is hardly a given. Benchmarks exist in a vacuum, not the real world. But from a strictly academic perspective, the Snapdragon 845 does look like it will bring improvements to speed and graphics performance overall when properly utilized, if Qualcomm's reference platform is any indication.