From fake reviews to boost rankings, to early reviews on unreleased products, the desire to rank well and get in there first has led to a drastic decline in the quality of online reviews. We investigate this worrying trend.
Reviews That Aren’t Reviews
A review should be based on a calculated assessment of a final product, only when someone has had a substantial chance to explore the product and all of its features. Someone forget to tell that to T3.
It’s impossible to review the Nintendo Wii U at least five months before release, especially when the games used are Beta versions of its games. A busy conference is also not an ideal environment to test out hardware. Michael Rundle sums this up best:
He’s right. Being able to post the first review of a product has become a driving force for online publications. People love to search for “Product Name + Review” as a query, and so being able to have a historic page with lots of value, is essential. The first review will get lots of shares and links, and then be in a very strong position when people search.
The Race To Be First
If the T3 article jumped the gun, then PC Advisor jumped the shark, with their review of the Wii U which was posted in June 2011. Almost a year and a half before release.
At this stage of release, the Wii U had no games to show, only tech demos. It also had unfinished hardware which has since changed. Every single conclusion drawn in the article is based on an incomplete product, and games that have since been updated.
However when you search for “Wii U Review” it’s one of the top results. It outranks many of the “reviews” written this year, and much of that is because it is a strong historic result. Google has no way of knowing that it’s fluff content designed to pull in traffic. It paints a bad picture of the product and could put people off, even though the review isn’t based on the real retail product.
What About Brand Image?
Due to this, it’s unlikely this is something Nintendo would encourage, and you’d certainly assume they would be against reviews of early release products?
Well, another Page One result for Wii U Review is… The official Nintendo Magazine!
So the magazine designed to represent Nintendo also published a “review” of unfinished hardware and software. Interestingly this review is far more fair and accurate which if you’re going to publish a review early seems like a better approach.
Yes the official Nintendo magazine is impartial so this probably isn’t a deliberate strategy by the brand. However, it would certainly be worth Nintendo adding a page to their own website with “Wii U reviews” that they could use as a placeholder for when real reviews launch.
How Can This Be Stopped?
As the Wii U was shown at a trade show with thousands of people testing it, it would have been very hard for Nintendo to control who saw the console and when they chose to write about it. The film industry however seems to have the right idea, using embargoes and private press screenings to tie reviewers in to contracts. These commonly work as “If you want to see our movie you can’t review it until …”.
This works to a degree, helping to ensure that reviews are based on final copies of movies and that they all land at a similar time helping generate a buzz around each film.
It’s by no means a definitive solution though. The New Yorker broke an embargo on The Girl With The Dragon Tattoo which led to the reviewer being banned from future press screenings. Depending on the contract you sign an embargo, this could even lead to a hefty fine, making them a very powerful tool to control the media.
In addition, embargos only apply if you’ve signed the paper. A blogger who reviewed an unreleased Hasbro product found himself under threat from lawyers and “representatives” who turned up at his house. However he bought the product from a region where it was on sale (so didn’t break a street date) meaning no laws were broken, and was free to review it how he wished.
Reviews Shouldn’t Be Rushed
The other drive behind embargos is that companies often want reviews to go out as close to release as possible, sometimes afterwards. This stems from a fear that reviews may be bad and put people off (another consideration to the constant searches for Product Name + review), as well as a desire to control reviews so they land at the same time for maximum hype.
However due to the fear an embargo may be broken that tends to lead to final products being sent to reviewers very close to the embargo date. Reviewers then have to drop everything, write up their notes as fast as possible and queue them up to be first when the embargo ends.
It’s a horrible approach to reviews
A reviewer should be able to spend as long as they need in order to write their views. Especially if that product takes a long time to fully enjoy (e.g. a book, a video game or anything that takes a considerable time investment to consume). Rushing through something in order to write the first review isn’t fair on the reviewer or the readers.
That’s why I like the way Kotaku does their reviews. They don’t always have them ready for the release date of a game, and wait till they have time to try a game fully. Whilst they lose countless page views and advertiser dollars by doing this, it does mean their reviews are trustworthy. They also say in each review exactly how much time they invested in each game.
Google Aggregators Make The Problem Worse
I really like Rotten Tomatoes and Game Rankings.com. Instead of reading individual sites for reviews, I generally look at the consensus on there as it’s often published after release and tends to be based on high profile real reviews. These two sites both get aggregation right.
Google on the other hand… not so much.
It started with local listings and citations. Getting more reviews on a Google local profile allowed it to perform better in the rankings. So in order to do well in local SEO you had to drive lots of reviews to your profile.
That led to people gaming the system with fake positive reviews for themselves and fake negative reviews for competitors (so their listings would look bad). It’s not exclusively a Google problem. Yelp and Trip Advisor are two high profile sites often marred with fake reviews.
If not for Google local listings, these sites would both have far less fake reviews. It also doesn’t help that Google can’t seem to add up review scores, unfairly favouring positive reviews over negative ones.
Where Do We Go From Here?
First of all, I love reviews and I feel they are an essential way to gain opinion on what to buy next. For five years I wrote reviews for various websites, so to say that we should stop all reviews would be both wrong and hypocritical.
Ultimately, as long as people are breaking embargos and early reviews continue to rank and drive traffic the issue will continue. As long as fake reviews can go undetected people will keep doing it.
For consumers my main advice would be to find a website that you can trust and stick with it. Think of products you like, and don’t like and then find somewhere that reviews them with similar thoughts to your own. Then stick to that reviewer and follow their work. You don’t have to agree with everything they say, but if you can trust that they’ve really tried a product and gotten to grips with it, then that’s a good indicator of how much you can trust them.
Ignore sites like Amazon as a review source. It’s really easy to write fake reviews (such as one book that got 200 fake reviews in 2 days, which still remain in place) and hard to tell whether someone who likes a product agrees with your own opinions.
When finding sites with reviews try to choose those who use the full spectrum of a score. If they rate from 1-10 then check to see if they have reviews at the lower end of the scale. If a website gives everything a good score then it won’t be reliable source.
Alternatively, scrap review scores all together and make your own mind up after you have read the content of a website. With the Wii U Reviews above, it’s hard to get any real insights on the console, and I wouldn’t base a purchasing decision of them. When I find a real review, it should be clear from the text that they’ve actually given the console a good work over. If a review has little depth, it’s unlikely to be a good guide.
Reviews as a medium are beginning to appear dated, and they hold far less authority than they did in the past. By reading multiple reviews of a product, and not relying on Google to find the best review customers, you can piece together a far more accurate analysis of whether a product is right for them. Trust the power of public opinion and your own views not the top Google result.
Blank gravestone via BigStock