Way back in the day, if you wanted to know if a piece of software was safe, reliable, or usable before you opened the package, you were out of luck. Once you rip that shrink wrap, you're stuck with it. The app markets of today are, by comparison, a breath of fresh air. A litany of user reviews let you know up front if a piece of software is crap or not before you download. Unfortunately, sometimes developers, like Noosoft Games, abuse this system by, as an example, using the Mechanical Turk system from Amazon to pay for 5-star reviews.

What Is Mechanical Turk

For the uninitiated, here's a little background: Amazon's Mechanical Turk system (named after an 18th century chess-playing automaton) is a platform for businesses to farm out minor, tedious tasks (called HITs) without hiring full-time employees, and lets users perform those small tasks for a pittance. It's not going to earn you $500 a day working from home like those email ads are promising, but it's a handy way to make some spare cash while proofreading some crap that a company doesn't want to hire editors for.

How It's Being Used

Left: Mechanical Turk HIT for app reviews. Right: Presto! Perfect 5.0 rating!

The advantages are obvious: cheap labor and easy money. However, as one of our eagle-eyed turking tipsters (turksters?) pointed out, developer Noosoft Games submitted the task of rating its app 5-stars and leaving favorable reviews. This isn't necessarily evil in itself. It's not like users are prohibited from actually using the app and providing an honest review. However, at $1.15 for 30 seconds worth of work, it's hard to imagine too many people will take the extra time, or turn down the HIT if they believe the app is crap.

The above HIT has since been removed (hence the hasty screenshot), but even offers a $0.20 bonus for a "successful" result. That means that, for the low price of $150, a company could buy over a hundred 5-star reviews. Needless to say, this creates some problems for users looking for honest experience. Since this isn't an automated system, it's harder to spot the usual tells of fake reviews: bad grammar, identical sentences from multiple users, vague reviews with no useful information. These reviews are written by real people with a bit of time on their hands.

2012-04-02 11h50_35

This game, The rise of the Aliens, has 121 5-star reviews, one 4-star review and two 1-star reviews as of the time of this writing. It's rare that a game maintains such a high ratio of positive reviews. It's not impossible, but even Angry Birds, the world's most popular mobile game, has only managed an average rating of 4.7 stars on its Rio incarnation on the Play Store. Angry Birds and Angry Birds Seasons are slightly lower. These numbers are pretty obviously inflated.

What To Do About It

For starters, keep in mind that this is just a shady promotion move. Full disclosure: I never downloaded this app and as a general rule, I wouldn't advise anyone else to, either. It should still be remembered, though, that using shady tactics to get reviews doesn't necessarily make an app unsafe. It does, however, call into question the motivations of the developer.

If you see something like this report it to Google/Amazon immediately. Google's official developer policies clearly state that developers "should not attempt to change the placement of any Product in the Store by rating an application multiple times, or by offering incentives to users to rate an application with higher or lower ratings." Which, by the way, also means that app that requires you to rate it 5-stars to unlock new levels/items is also in violation of Google's policies.

The Mechanical Turk policy also specifically prohibits HITs that require users to download software, which the Google Play Store requires before a review can be posted. This means that these tasks not only directly violate Google's policies on Play Store behavior, they also violate Mechanical Turk policy.

A quick search on the Mechanical Turk system doesn't return any similar HITs right now, but that doesn't mean everything is in the clear. Amazon's system isn't the only one out there that provides services like this, nor is it hard to imagine that this kind of shady dealing could be done without the need for a central platform. It's also worth noting that this same type of thing could happen on the iOS App Store, Windows Phone Marketplace or with any other system. Since this tactic involves directly paying a real human being to write a review, it's virtually impossible to distinguish from a legitimate review.

The best way to protect yourself, then, is to keep your wits about you. Remember that nothing online is universally liked. If you spot an app with crummy-looking screenshots, but a thousand reviews stating it's the best thing ever, something might be up. And if everyone says it's amazing, you might be getting played. It doesn't matter how good an app is, someone on the internet will always hate it.

As always, if you find something suspicious on the Play Store or anywhere Android-related, hit us up at tips@androidpolice.com. We are here to serve and protect, after all.

Thanks, Denise!