The Great Review Debate: Can we find our way back?

This holiday review period has brought us "h8/10," journalists griping over review code and the concept of pre-ordering pushed to the limit. How did we ever get to this point? The publishers have the answers, says Patrick Garratt.

Put your feet in the shoes of the PR and marketing departments at a large publisher. You've got your game, and you've worked on it for about a year. You're just as aware of the weak areas as you are of the strengths. You know there are elements sure to be picked on by journalists and you're expecting 8s, the odd 7 and a few 9s. The developers and top brass are expecting 12s.

The holiday season brings with it the year's biggest game releases, and some kind of review controversy is always in tow. This year, however, the entire process of reviewing games has reached an impasse as the industry polarises between triple-A and "everything else," with sky-high expectations on certain products making for farcical situations surrounding scores.

In addition, journalists have been constantly griping, both behind the scenes and on social networks, that exclusive reviews and deliberate stalling of code delivery is creating a difficult situation both for those buying games and those seeking to have a review live for launch day.

To understand why game reviews are handled in the way they are, and why publishers seek to exert so much control over when reviews go live, it's necessary to face a few facts.

Firstly, a games publisher's entire reason for being is to promote and sell video games. Once you accept this, all of the behaviour you see surrounding reviews from journalist, developer and publisher perspectives becomes crystal clear.

Secondly, publishers will do anything they possibly can to counter the cancellation of pre-orders. Pre-orders are vital for the success of any bigger game these days, before word of mouth and any negative reviews have the potential chance to stop people buying it in the weeks following release.

And thirdly, we have the continuing presence of Metacritic as a metric for creative success.

There's no fourthly. Let's take a look at some of the aspects of game reviewing journalists and gamers constantly moan about and explain why they happen, taking the aforementioned truths into account.

Exclusive!

As it's the first one you're likely to see, let's think about the exclusive review. This is arranged between an outlet and a publisher. The publisher will never agree to this if there's a risk of a low score, for obvious reasons. That's not to say the publisher will demand a certain rating; the publisher in this instance, and the outlet involved, will be well aware of what it means to have this happen. You're looking at a 9 or higher here.

An exclusive review for a multi-platform game is normally reserved for mid-spend games from third-parties. Dark Souls is a topical example. IGN went live with an exclusive review days before the general embargo. It got a 9. The rest of the world gave it a 9, too, so there was no danger there, and IGN did a 24-hour play of the RPG leading into the review, which was probably part of an overall deal.

Why would IGN want to do this? Traffic, obviously. Why would the publisher want to do this? Positive promotion of the game directed straight at the people most likely to buy it. Everyone searching Google for "Dark Souls review" in the days before the general embargo lift would land at IGN, and would see a high score. To enter into this agreement both parties would have to be confident that the game was of a very high quality. Keza MacDonald wrote the review; she was a big supporter of Demon's Souls and, as I'm aware of first-hand, wanted to play Dark Souls more than being able to ingest food for most of this year. The deal worked for all parties.

Exclusivity in this way doesn't have to be limited to a single site. Publishers may offer regional exclusives, print exclusives, online exclusives or platform exclusives. They will try to work with people they trust and journalists they know appreciate the product. The reason for doing this is always the same: it's so the first critical contact a consumer has with the game is a positive one.

The Dark Souls exclusive is an instance of a deal working for everyone, very much including the consumer. It doesn't always. It's likely that larger sites wouldn't enter into something like this if they knew the game wasn't up to scratch, but a publisher is still likely to try. And if the larger sites don't want anything to do with it because it may damage their integrity, there are plenty of other outlets that will get on board because they need the traffic or magazine sales. The danger is obvious: you get a skewed score in return for exclusivity. This is why people pull a face when an official mag comes out first with a review and gives a very high score. Magazine covers are even more vital to the survival of the publication than exclusive reviews are to websites. If publisher PR can get a decent cover with a good score and agree on an exclusivity window, all parties are likely to agree. There's a grey element here, though, and people aren't that stupid.

The comments are always the same: wait for the general reviews.

Where's my code?

Which leads us on to the general reviews themselves. Again, to understand why this process works in the way it does, it's important to think like a publisher. The review element of a game's life-cycle is only a part of its customer-facing promotion, and is the most difficult for the publisher. Note I said "promotion": if you're a publisher, the review is not a completely unbiased appraisal of the game; it's a chance for you to sell more copies.

Put your feet in the shoes of the PR and marketing departments at a large publisher. You've got your game, and you've worked on it for about a year. You're just as aware of the weak areas as you are of the strengths. You know there are elements sure to be picked on by journalists and you're expecting 8s, the odd 7 and a few 9s. The developers and top brass are expecting 12s.

First of all, you create a pre-order campaign. You'll have to judge the level of fandom here. If the IP's got a rabid following you can go large, as Activision did with Modern Warfare 2 or Namco did with The Witcher 2. Big package, high price, silly excitement, large mark-up.

If you've got something a bit newer, you might offer a lesser, cheaper premium package which is designed to make people think, "Why not? It's only an extra tenner and I get the blue multiplayer jumpsuit of Quarg on day one!"

When you pre-order a game you are buying it based on an image that is completely controlled by the publisher. All you've seen of the game at this point is some screens, a few edited trailers and maybe the odd interview. This is why publishers do it. It's cash in the bank based on the best possible perception of the product.

Your pre-orders have gone well. Execs are saying things like, "Gungo-F**kers 3 is tracking well ahead of expectation in qwarder four" in fiscal calls; their optimism is based on the level of pre-ordering generated by PR and marketing. Loads of pre-orders, guaranteed great initial sales, happy investors.

Then, though, you have a problem: you have to stop people cancelling their pre-orders for any reason. This is vital for the publisher, and it will usually do everything possible to protect the game from being shown in an uncontrolled way before release. Pre-orders really can make or break a game in today's market. Publishers are in the business of "make".

As an example, try going to somewhere like the Eurogamer Expo or PAX and filming a game off-screen. You'll be immediately told to stop. Why? Because the publisher doesn't want the footage appearing online at a crap resolution, and risk damaging the perception they're trying to foster that Gungo-F**kers 3 is the greatest piece of software ever committed to disc. Some publishers are fiercely protective of footage appearing on YouTube, for example, and will have any videos removed instantly.

See how important pre-orders are if you're a publisher? And if unauthorised online footage presents an issue for pre-order retention, reviews are a absolute nightmare which must be controlled. Honesty, when you're trying to sell something, may not be the best policy.

Which brings us onto the second favourite gripe we see with reviewing: the timing of review publication and the availability of pre-release code. There's a perception that anything below a 9 for a core title isn't going to cut it, especially in the triple-A space. If you suspect scores may fall below that, you have to manage publication timing. If a load of 7s go live a week before the game's released, there's a good chance a ton of your pre-orders will cancel before they ship. And that, as any publishing executive will tell you, can't happen.

You manage this in a few different ways. Firstly, you refuse to allow outlets review code unless they sign an NDA saying they won't say anything about the game, at all, anywhere, until launch day. These NDAs now include clauses about Twitter and Facebook: if you sign, you really can't say anything in public about the product itself.

With the proliferation of social networks, we've seen an explosion of people tweeting images of the review code disc itself, or the press kit, or whatever else, with a note saying they've signed an NDA and can't talk about the game. As a reaction to this, the most recent addition to the NDA we've seen is the "super-NDA". This NDA stops any mention of the fact an NDA has even been signed. Once you've got a journalist with their name on this sucker, he's yours until embargo.

Secondly, and even if you've put NDAs in place, you send the code at the absolute last minute. This cuts down the risk of any media or impressions leaking before the cut-off for pre-order cancellation. VG247 staff regularly have to sign final code NDAs, despite the fact we don't publish our own "reviews". We do, however, capture video footage, do voiceovers and write opinion pieces on code, all of which could have exactly the same effect as a review score.

This is why you get reviews on launch day, and not weeks before so you can "make up your mind" as to whether you might want to buy the game. The only decision the people involved in creating, marketing, PRing and eventually selling this game want you to make is that you should buy it. Think like the publisher and it all becomes clear.

I still know journalists who are baffled as to why they haven't received games in good time to have a review published for launch day itself, or even to be live well before launch, so people can make an informed purchase choice. Ask yourselves these questions: do you work for a popular publication? Are you known for "honest" reviews? Could you affect week-one sales and even pre-orders with stand-out negativity?

That's why you don't have the game.

And before you all start having a little fanny about any of this, get some perspective. As a journalist, you are not "entitled" to be given a company's property before it goes on sale and potentially wreck its chances at market. Publishers do want reviews up for launch because it's a norm in the games world and looks suspect otherwise, and they want scores live for consumers looking to make their minds up on launch day, but they need high scores. Some games may have a huge amount of reviews live on embargo, some less so. This is the publisher manipulating the review spread, working with publications it believes will deliver good scores for launch. Why are you remotely surprised by this?

As a gamer, no one is forcing you to pre-order any video game, or even buy it at launch. You are being marketed at. It's your money and you can do what you like with it. Publishers buy ads on sites like VG247, IGN, Gamespot, CVG, Eurogamer, Destructoid, Kotaku and all the rest because they command large audiences of video game enthusiasts. That's you. You probably already want to buy the game. You want to be excited about it. You want to talk about it. You want information on it. You want to know the release date and what you can get for pre-ordering. You're a perfect target for "enforcement" marketing, and, in all truth, you'll probably enjoy the game in question regardless of whether or not it gets an MC average of 72%, 84% or 92%. You're a hardcore, knowledgeable hobbyist and you've got a good idea of what you're getting yourself into when you hand over your cash.

You have a choice. If you've been bitten before or you're just more careful with your money, wait until the game's launched, do your research, find some reviews published after the embargo, let the game get patched a few times and then buy it (or not, as the case may be). You're allowed to do this. You are an individual. This is why you have a brain.

h8/10

This situation is not exclusive to games releases, but exists with product reviewing in any sector. In the eyes of the publisher, we're part of the process of marketing video games, and this is why the entire foot-stamping spectacle from journalists and gamers is so ridiculously naïve. Did you see a review of iPhone 4S before it released? Of course not. Be Apple for a second: why on earth would you take the risk? Why do you think Apple went so beserk over Gizmodo's iPhone 4 exclusive in 2010? It's because early, uncontrolled access to products can impact sales. Nothing more or less. Exactly the same thing is true of any marketed product, and that includes games.

What is new this year in the games industry, though, is the level of expectation on certain products producing worrying attitudes around scores. Both of the most acute examples of this have been brought on by Eurogamer reviews this autumn, the first for Gears of War 3 and the second for Uncharted 3.

If you're a review journalist you can pre-order the game yourself, get it on launch day, play it, write a review and put it on the internet. Radical, I know, but it can be done.

Eurogamer's Johnny Minkley gave Epic's latest 8/10. I interviewed Cliff Bleszinski the day after the review embargo lifted, and he jokingly said Eurogamer was staffed by "haters" as a result of the score. He went on to say he believed the third game was the best in the series and was "upset" that EG had rated it lower than Gears of War 2's 9/10.

I can't speak for Cliff, but I believe he only used the word "haters" because it's internet-speak. Regardless, though, he was clearly annoyed by it. In his eyes, an 8/10 simply wasn't good enough.

Following from this was Simon Parkin's 8/10 for Uncharted 3, again on Eurogamer. This really did kick the hornets' nest. Jim Sterling's latest video for The Escapist has a look at what happened on sites like NeoGAF in the wake of the review being posted, and even David Jaffe threw his weight around in support of Parkin, such was the severity of the pushback.

Sterling himself has been at the centre of this type of hysteria this year, after giving Batman: Arkham City 8.5/10 on Destructoid. Watch the video to get a steer on the type of reaction this drew from gamers.

The reason gamers get so angry about a game they've been waiting on for the past year is obvious: they've been invested for months, and they want to see a 10. Why? Because they hope the game they're about to play is "perfect". Anything below a 9, especially for one of the big platform-specific games, is now viewed as a low score. Oh, the irony: back in the day, we used to combat the notion that 7/10 was an average rating, when it was, in fact, 5/10. Giving a game a half-mark now is akin to saying it's only worth buying if you need something to use as a firelighter in the cold winter months.

The situation has clearly reached an absurd level if scoring a game at 8/10 is deemed unworthy. How did we get here? Again, think like a publisher for the answer.

Metacritic is pretty much the only hard measure of critical success for videogames. It is used by all levels of the games business – including the media – as a metric for whether or not a game is "good". Publishers like EA base entire strategies around lifting Metacritic scores across their portfolios, and investment execs like Michael Pachter routinely quote MC averages as a rule of accomplishment.

Pachter tells people where to put huge amounts of money.

This need for a high MC score by senior executives at publishers pushes down on entire corporations, right down to PR level, where they interface with the media. And it's this desire, this need for very high Metacritic marks that has led to a culture where games that carry sub-9 scores are no longer seen as true hits. And everyone is now aiming for true hits in an environment where disc-based, double-A games are being mercilessly slaughtered. In turn, the gamer community now expects the likes of Battlefield 3, Modern Warfare 3, Uncharted 3 and all the other threes to be 90-plus by default. When they're not, we see a great deal of friction.

This is, of course, a completely manufactured situation based on corporate politicking. The result is that we may have allowed ourselves, as an industry, to arrive at a point where reviews scores are essentially meaningless. If an 8/10 doesn't say a game's good, what does that say about the field of game reviews as a whole?

Fortunately, the situation's not hopeless. The media doesn't need to be part of the publisher's review process. If you're a review journalist you can pre-order the game yourself, get it on launch day, play it, write a review and put it on the internet. Radical, I know, but it can be done. Maybe your readers will trust you more if you focused on reviewing the game after the fact, in a control-free environment with all online modes working in the wild, instead of working by the publisher's rules, attending "review events" and insisting on having a launch-day score.

If you're a gamer, and you're seeing an 8/10 as a h8/10, maybe you should just get some perspective. Why don't you play the game and see what you think? You never know: you may just enjoy it despite the number and learn to find reviewers that work for you instead of screaming into the void over arbitrary metrics.

And if you're a publisher? If you're a publisher, perhaps you should stop placing such a ludicrous emphasis on review scores and concentrate on making the best games and marketing them in the best way. Focus on excelling instead of controlling. There's been a lot of talk about gamers calming down over scores, but you'd be amazed at the sort of reaction a "bad" score can cause at a publisher. Maybe, Mr Publisher, if you weren't trying to constantly balance the need for early reviews and high day one results with a façade of press independence then none of this would matter so much. Maybe having a day one review isn't so important after all.

Maybe. See you this time next year.