There is a long standing disagreement among many gamers as to how reviews should be written for massively multiplayer online games, or MMOs as I will henceforth refer to them. There are a variety of reasons for this, but many of them stem from the fact that MMOs are made to be played for a long, long time.
One issue is that the average MMO player will put hundreds to thousands of hours into their MMO of choice, yet for professional game journalists deadlines are a fact of life and simply cannot put in the hundreds of hours necessary to adequately assess and weigh in on an MMO and still get a review out before everyone who cares has already read reviews from all the competitors. In most MMOs, the first 20, 40, or even 100 hours play very differently than later on, once the higher levels are reached. If a game's lower level gameplay is all flowers and gumdrops, that's all well and good but how will the game keep the experience rewarding later on? On the other hand if a game is dull or merely slow in the early stages, it still might ramp up after 20-40 hours once the player begins getting in on high level activities like raids and is powerful enough to access new areas and better quests. Of course it might just be a snore inducing feat of boredom straight through.
So how does one review an MMO?
Should it be reviewed based on the game exactly as it is at launch or should bugs, exploits and other issues be ignored based on the assumption that they will be fixed in the future? Should a reviewer have to guess whether the game will build up a thriving community and economy?
In my opinion an accurate MMO review would have to be written about four to six months after launch and the reviewer should make it at least 70% of the way to the highest level in the game if not more. Unfortunately for mainstream video game press that isn't an acceptable option, not to mention that most people who were interested in the game will have already either tried a free trial or read one of the early reviews. MMO-specific websites sometimes put out late reviews after putting loads of time in, but those must be taken with a larger grain of salt than usual because they are often written by people who are part of the game's fanbase.
Another possible answer that I've heard is the idea of re-reviewing a game from time to time or after major updates. It's a fantastic idea but there are two major problems as I see it. The first problem is that as it is, gaming publications and websites are already swamped trying to find time for their journalists to research and write previews, set up interviews, play through the games that they have to review and then write the reviews. We're not talking about movies here that only need to be watched for an hour and a half to three hours. Games take anywhere from ten to upwards of fifty hours to play through for a proper review. Try adding in the responsibility of re-reviewing games on a regular basis and it will just be too much. The second problem I see with re-reviewing games is that if re-reviews become the norm, then developers and publishers will be that much less motivated to make sure a game is adequately complete before releasing it. If an MMO is going to get re-reviewed anyway then why not just release the beta version in stores and say it's done. It can always be patched for a higher score. A similar mentality is already on the rise on the console game side. Ever since the current generation of consoles has allowed games to be updated, I've seen a lot more games being released in an unfinished form.
And what about other games that need time to 'grow'
Soon, we'll have a whole new style of game that may raise similar and possibly more questions regarding how reviews should be written. LittleBigPlanet is set to come out in October this year and the game is essentially a super-intuitive game creation tool. The game itself almost mirrors the Playstation 3 in the manner that the creators have created a fantastic and powerful tool, but since the people who use it won't be able to instantly grasp everything it is capable of, the most delicious fruit that it will bear won't show up until a ways down the road. As it has taken Playstation 3 almost two years before games were able to show what the system was really capable of, (largely due to developers learning how to program for the system), it may be quite some time as well after the game's launch before we see the really good content for LittleBigPlanet.
So how does one write a review for such a game? If the single-player game that comes bundled in is amazing and the design tools are fun and intuitive then obviously the answer is easy, you score it nice and high, discuss the game as-is and mention future potential. On the other hand, what if the game as it is at launch is simply an awesome game design tool with an afterthought single player game bundled in? Do you review it based on the promise of great community content popping up in the future? Can a review truly be written based on the potential of a game rather than what is actually there?
So undoubtedly now you're looking for my answer to all these problems. Truth is, there's no 'right answer' and any way that a review for LittleBigPlanet or an MMO is handled, it will be forced to make assumptions and walk a fine line between not giving a game enough of a chance and giving a game too much credit. As more non-traditional games are released that -by design- won't reach their innate potential until long after they have been released, (hopefully many more), it will be interesting to see how reviews are handled going forward.