Do game scores get inflated?
OPINION
Inflating game scores. We all know that it happens but do we always understand the reasons behind them? Do we know how often it happens and how big the problem really is? We try and find some answers for these very good questions.
But… we know it happens. If even only half of the rumours are true, then the problem is clearly real. There is simply no denying it. In fact, there is pretty solid proof of it happening as Gamespy altered a review score back in 2005, causing quite a scandal in the process. And there are other indications of scores being inflated; user reviews that often turn out to be significantly lower than the scores given by reviewers.
A good example of the opposite is an article on Curmudgeon Gamer that was published last year. It compared Gamespot's review scores with the scores that Gamespot's own readers gave to the same game. In it, the author of the article showed that on average, Gamespot scored their games 0.3 points lower than their visitors did. It is only natural for reviewer scores to differ from user scores, but a difference of 0.3 when comparing thousands of scores is quite significant.
Now, this discussion is as much a fact of the gaming life as the question of whether or not you want to bash your neighbour's head in after going head to head in Pacman, but somehow it is really starting to get to me. Before I continue, I should make it very clear that I have been reviewing games for about 4 years. During this time, I have heard the rumours, seen some proof here and there, and reluctantly, I have come to the conclusion that yes, it does indeed happen. However, the reasons for it are not always as obvious as one would think.
Let us look a little closer at the GameSpy case. The reviewer in question obviously didn't like what he was playing and scored the game negatively; a 1.5 out of 5. The editor, however, felt this did not do justice to the game, changed the text a bit and slapped on a new score; 3 out of 5. From experience, I can tell you the editor will not have done this lightheartedly. He will have talked to his colleagues and will have done a lot of soul-searching before taking this action. Did he do it to please the publisher? In this case, I seriously doubt it.
If you think about it, 1.5 out of 5 is pretty low. It is important to note however, that game reviewing is all about personal opinion. If you check the game's average score on GameRankings then you'll see that the average reviewer did not agree with him (the average score is 71%). One could argue that this person is so far off the average that he should consider a change in career. However, his job is to write his opinion, not to write what the average person might think.
Ah, hit a nerve there have I? Yes, I agree, a reviewer's main task should be to inform his readers in such a way that they can decide for themselves whether they should buy the game, or leave it on the shelves. However, reviewing isn't a science and it is not about being objective. It is about emotion and senses. It is surprise, disappointment, joy, discovery, elation, expectation and more. It is also about having a bad hair day and not feeling too well. It is about a hamster that kicked the bucket that very morning and being in a lousy mood because of it, and it is about having had sex with that girl you've been chasing for three months who finally caved under your charms. To sum it up: a reviewer is only human and you will never, ever get a 100% objective score out of him.
Before I digress too far, back to the editor inflating the score. I cannot recall ever having done this with any of our articles, but I can fully understand that sometimes an editor does not agree with a score. His responsibilities to the magazine/site are different than the author's and in the end, it is his call and one that -under circumstances- I can agree with.
So what about inflating scores to keep the publishers happy? Yes, that happens too. But to be honest, I don't think that big sites like IGN, Gamespot and Gamespy need that. They will have considerable sway over the publishers and can probably get away with even threathening to score all of EA's games very low to stave off complaints about low scores. Small sites hardly get noticed at all by the bigger publishers and for them, inflating the scores makes no sense at all. Why try and please someone you're not even in contact with? So we're left with the medium sized sites. Does it happen there? It probably does but not on a large scale.
Does all this affect your buying? I seriously doubt that. The hardcore gamers among us will buy the game without reading a single review. The rest will wait until they have had a chance to read a couple of reviews, enough to make up their mind, and then buy the game. Also, I think gamers are clever enough to know when a wesbites' scoring is consistently out of whack with their own thoughts about the games they play, and will just go elsewhere. God knows there are plenty of review sites out there.
That brings me to why this discussion is really getting to me. It is what they call a moot point. A gamer will go his own way. Sure, he'll check out the scores on sites he's comfortable with and trusts, but in the end, he'll buy the game, inflated scores or not. If a game he decided not to buy is worth it afterall, his friends will tell him and he'll go out and buy it. Someone who is really interested in the game and trying to make a decision, will actually read some reviews and read between the lines. By doing so, he will filter out the comments by the reviewer that do not apply to him and focus on the ones he deems important.
And the reviewer? Well, he just does his best to inform you as well as he can and to the best of his conscience. He will do so knowing full well that others may not agree with his judgement at all but also that we can all agree to disagree. Afterall, whether or not you like a game, is just a matter of taste.
*update*
Barfo commented correctly that I had switched the scores around in the paragraph about the article on Curmudgeon Gamer. While it changes that paragraph, the conclusions made in the overall article remain unaffected. Thanks Barfo for setting me straight. :)