Friday, September 19, 2008

Gaming by the Numbers: The Case for Dropping Review Scores

Mario Kart Wii is a better game than Assassin's creed, at least according to

I was once a big believer in review scores, but I think it is time to move away from them. There is nothing inherently wrong with an author reviewing a game and providing insight into what they perceive to be its positives and the negatives. However, trying to distill the quality of a game into a single number is a shallow method of analysis that serves to limit the development of thoughtful analysis by diverting people towards petty arguments.

In my view, there are at least three major reasons to abandon video game scoring systems:

1. Games Can Change, but a Number is Forever

Not to get overly "critical theory"-y on everyone, but the video games, like any other piece of art or entertainment, are defined both by their creators and their consumers. The line between creator and consumer is easily blurred when players begin to utilize features within games that the designers may not have consciously created. Because of this, giving a game a numerical score exacerbates the already-glib nature of a review.

Consider the way in which gamers institute modified sets of rules in games: doing a "speed run" in Super Mario Bros., trying to get through a Zelda game with as few heart containers as possible, or attempting to play the Sims by building a house with only one room. None of these game modes are detailed in the official manual, and each arise from the player's modifications.

In the PC world, game modification has been taken to an extreme: who could have guessed Team Fortress would come into existence when Quake was being reviewed, and how would this have effected its (already very high) scores?

Finally, some games (I'm looking at you Will Wright) feature gameplay that emerges only after a sustained period of playtime. I think it is absurd that Spore is being scored right now, in light of how much of the game depends on player input. Complaints about the weakness of herbivorous characters may be rendered moot when veggie-players band together and create a vast Spore-alliance dedicated to ruling the Spore-universe. Putting a number on a game a week after its release is a shortsighted way of viewing a game that ignores the dynamic nature of art.

2. It's Like Comparing Apples and Machine Guns

As I alluded to in the initial Mario Kart/Assassin's creed example, it is unproductive to use the same basic scoring system on games in vastly different genres. This method of evaluation is unnecessarily focused on standardization.

Different genres provide unique forms of gameplay whose merits should be evaluated in their own right. The basic question a review is trying to answer is: "Is this game enjoyable?" The best explanation, regardless of the answer, would focus solely on the elements of the game without trying to quantify them for mindless digestion.

Mario Kart Wii and Assassin's Creed both succeed and falter at certain points, but trying to convince me that one deserves a higher score than the other is like arguing that chocolate tastes better than bacon. The enjoyment a game provides is a novel product precipitated by the interaction of developer and player. Slapping a number on that experience does a disservice to the player, the developer, and the game by inviting faulty comparisons and oversimplified judgements.

3. Zealots Feed on Numbers

A score's most damaging effect is the adversarial mindset it fosters. The attitude of trying to find "the best game" is a destructive practice that produces discord within the gaming community while also undermining the broader cultural credibility of video games as a medium.

There is nothing worse than someone hell-bent on blindly defending a game towards which they feel some misguided personal attachment. Any criticism is defended with bellicose rhetoric, ad hominem attacks, or good old-fashioned crappy logic. Numbers are simply ammunition for these people, who gleefully jump on a lower-than-expected showings as proof of a failure, or continually trumpet a perfect score as evidence that the epitome of gaming has been reached. Figuring out whether Call of Duty 4 or Halo 3 got higher marks does not make one "teh suckz," it simply creates trivial arguments that distract from thoughtful analysis of each games' respective components.

The inclination towards "scoring" an artistic works seems weaker on a broader societal level than it does in the gaming world. Many newspapers and magazines have dropped numerical or "star" rankings for films. Similarly, have there ever been scores for literature reviews? The most respected art, literature, and film journals do not try to boil down their analysis into a number, yet the most respected video game journals do.

This is not an indictment on the leading game journals, but rather a call out to the entire gaming community. Ultimately, the big game journals represent the state of the community, a state in which scores still have currency. It is time to change that, to embrace the open-ended quality of games, to analyze them on their individual merit, and to cease the bickering that ensues when we make vapid judgements based a fallacious hierarchy of game quality.

It is time to stop playing the numbers game.


  1. I understand why it happens but I have always found it interesting that people rely on and dare I say it, expect scores to be so important to their buying decisions or as ammunition for their arguments.

    I have always read reviews with the intention of finding out about a game, what it does and doesn't do and so on and I have never really paid attention to the scores included with the review. Perhaps I am different than most readers, but actually reading about the game has always given me a better idea about the game than a simple number has.

    That said, these days I find it hard to read reviews and to be honest, I think I am just over them. Not just reviews either but the whole preview/review process that most games go through. I can't exactly pinpoint why I am over them, but what I do know is that I definitely prefer reading the more in-depth stuff, be that an interview with developers that doesn't have the typical PR answers, a feature on a game or franchise or say a blog post that focuses on maybe one aspect of a game rather than trying to summarise the entire thing at one time. I rarely visit the big websites for gaming content these days, instead I stick to Kotaku for news and blogs and maybe even (some) forum impressions to give me an idea of whether I will like the game in question or not. I also love just watching people play (via Youtube or whatever)games as that can also give me a great idea of whether the game is for me or not (even if I don't realise it at the time).

  2. Warning: long comment ahead!
    I've been arguing this point for years. Here are some very good reasons why I agree with you:

    1 - I love Super Mario Galaxy. From the moment I fired it up, I was absolutely awestruck. From the gameplay to the level design to the music, I was just absolutely entranced every time I played the game. By the time I was finished, there was no doubt about it, this was my favorite game. When people ask my opinion about it, I tell them one simple thing: Super Mario Galaxy is the game that I have been waiting for my entire life. To me, it is the definition of a perfect game, and well deserving of the many perfect 10s that it got. That said, despite how I feel, is it really a 'perfect' game? No. There are a few things I didn't like. Things I thought were bad ideas, or just plain boring. (I'm looking at you Purple Coin Challenges.) If I had designed the game, I wouldn't have put those in there. However, that in no way detracts from what a great game it is, and if we were to use a scoring system, I'd have to give it a 10/10. that itself implies perfection on a game that is perfect, but isn't. Figure that one out.

    2 - I don't really care if Halo 3 got a 9.5/10. I don't like it. In fact, I hate it. In fact, I hate what it represents (in my mind.) I've never liked that style of game, so the high 'score' means nothing to me. The same goes with sports games. I don't care if Jesus himself endorsed Madden 09, I'm not going to play it. In that regard, the score is utterly useless.

    3 - Reviewing classic games seems to be all the rage right now, and I can't figure out why. It seems that a numbered scoring system is even more arbitrary when applied to classic games newly reviewed. How can you give one game a low score because you don't think that it 'aged well', but give your favorite Zelda game a 10/10 still just because to do otherwise would be blasphemy? For a sort of example, look at the newly released Mega Man 9. IGN gave the graphics a 3/10 for being 'outdated'. Are you people complete freaking morons? THAT'S THE POINT OF MEGA MAN 9! It is a phenomenal game. This is what gets me about reviewing old games. Why would you go back and give it a lower score just because technology has progress 20 FREAKING YEARS LATER!?

    That's just three reasons why a numbering system is bad. When I read a review, I just want to know about the game. Tell me what you think worked, and what didn't. Tell me reasons I should try it or avoid it. That's all I want to know. I have 0/10 interest in a numbering system.

  3. @nismo

    I share your unease with the preview/review cycle. I think the amount of hype behind games affects the way both "professional critics" and normal players react to a game. It's rare that a game can live up to our imaginations of how cool it might be.


    Nothing wrong with a long comment. As you can tell, Jorge and I often struggle with our own leanings towards verbosity. ;)

    Your point on classic games is an important one (maybe we should post on it...). As you say, the historical context in which a game is reviewed drastically affects its reception. Evaluating a game's current and lasting impact is quite a tricky business.