A Nintendo community
for the fans, by the fans!
 Go to forum index
Video Game Review Scoring Systems: A Comparison
Editorial by 
Editor
August 06, 2012, 00:13:47
 
This started as a post in the "IGN switching to a 20 point scale..." thread, but then my usual typing diarrhea started up and I decided to put it out as an editorial. There are a lot of scales competing for use amongst video game sites. This is just one man's opinion on them.


100 Point Scale

What you think it says: "Our sophisticated editorial staff can evaluate, with laser-like accuracy, the success or failure of a particular game."

What it secretly says: "Our editorial staff are fulfilling their childhood dreams of writing for late 80s/early 90s video game magazines. Particularly if this 100 point scale comes coupled with a box score that rates the sound and graphics separately. And double that if it comes with funny cartoon pictures."

Advantages:

- An individual reviewer could use this scale to rate their game preferences with pinpoint accuracy.
- It's an institution. The 100 point scale, based out of 10, with decimals, has been the default setting in gaming rags for so long that most everyone understands it, despite any obvious flaws.
- You can give a game a 9.9 and still think of yourself as being fair and balanced. "Hey, I didn't say it was perfect!"

Disadvantages:

- There's no way an editorial staff can agree on anything with pinpoint accuracy, so that whole premise is flawed.
- Even an individual reviewer is going to have a hard time telling you the difference between, say a 7.3 and a 7.4. At that point of precision, isn't the reviewer's mood that day just as important as anything in the game?
- Like it or not, everything below an 8 is going to be considered a failure by a significant chunk of your audience, even though that makes no sense.
- Your audience constantly debates whether the scale works or not. Have fun listening to that. It ain't going away.

Kris's Recommendation: Stick with this if you're already using it and it's working. Yeah, it's a charade and yeah hearing people complain about it is a pain in the ass, but chances are you're already a big box gaming site, anyway. If you have problems with your readership figures, it's probably not because you have an illogical rating scale.


20 Point Scale

What you think it says: "Our reasonable editorial staff knows that there are flaws with the 100 point scale, so we've come up with an original solution that everyone can agree on, one that gives us greater flexibility while still removing some of the more ridiculous characteristics of the 100 point scale."

What it secretly says: "Our unreasonable editorial staff can't agree on jack shit. No one's happy with this idiotic compromise, but it was the only way we could keep everyone from killing each other. Hope you like the number 8.5, because you're gonna be seeing a lot of it."

Advantages:

- Your editorial staff stops fighting and simply goes back to grumbling to themselves.
- The gambling pool for how your site will score a game just got a whole lot easier.
- 4 idiots in a small town in Nebraska think this is the best system ever. At least they're happy.

Disadvantages:

- 8 is still going to be the base level for success or failure with a significant portion of your audience. You've simply trimmed all the wiggle room above that.
- You still get to spend all your time insisting that a 7 is a pretty good score, even though no one believes you. In fact, with this new system, you get to have that argument more!
- Your entire staff looks like a pack of morons.

Kris's Recommendation: Start looking for a new job if everyone you work with agrees this is the best scale. Or pinch yourself. Hey, if you're lucky, you are in the Twilight Zone.


10 Point Scale

What you think it says: "Our pragmatic staff recognizes that larger scales are silly. We need plenty of room to rate games, but we're not going to try to snowball you into thinking we need 100 points of accuracy. We respect your intelligence."

What it secretly says: "Our sentimental staff just can't live in a world where Ocarina of Time doesn't have the number 10 next to it."

Advantages:

- You get flexibility without the illusion of precision.
- You can probably convince more people that a 7 is a pretty good score. I mean it's not like you can give every game an 8, 9 or 10.

Disadvantages:

- A significant portion of your audience still thinks you better give every game either an 8, 9 or 10.
- When the big box 100 point scale guys give a game an 8.5 and you give it a 7, you have to explain that those are similar scores, even though no one believes you. And you get to do this every time.

Kris's Recommendation: Well, you can sleep at night, anyway. But your scale is still too easily comparable to the 100 point scale you're probably trying to get away from. Plead that you've solved that problem all you want, but your logic doesn't work here.


5 Star System

What you think it says: "Our wise editorial staff understands that you look at ratings simply to determine if we recommend a game or not. You don't expect pinpoint accuracy. You just want the straight dope."

What it secretly says: "Our grumpy editorial staff hates rating games in the first place."

Advantages:

- FINALLY, there's a system that breaks free from comparisons to the silly 100 point scale!
- The system is easy to understand, particularly since it's been used for years by film critics in newspapers.
- Every point on your scale means something. You can give something 2 stars with no problem.

Disadvantages:

- You're going to be constantly tempted to shit out half-stars, which defeats the whole purpose of going to a 5 star system.
- You have somewhat sacrificed flexibility for having a logical system. Hope that helps when you can't decide whether to give a highly anticipated game a 4 or a 5.
- Half of your editorial staff will mourn for 6 months about Ocarina of Time not having the number 10 next to it. Some of them will try to kill you.

Kris's Recommendation: Consider yourself a hero. This is the most respectable scale to use, period. Catch me in a bar, I'll buy you a drink.


Up or Down

What you think it says: "I'm the Roger Ebert of video games."

What it secretly says: "I'm not the Roger Ebert of video games."

Advantages:

- Well, at least you cut to the chase.

Disadvantages:

- Look, unless you have the credibility in stock already, no one is going to listen to your Up or Down scale. Even Siskel and Ebert only made the system work by fighting with each other over it every week in defense of their views. And, even then, it was technically a 4 point scale, because the real endorsement was if the two them agreed and both gave the thumbs up.

Kris's Recommendation: Look at that guy in the mirror. See that guy? That guy needs some perspective on his place in the Universe. Maybe take up stargazing or bird watching. Resist the urge to rate the birds, though.

Or you can just yell at them. That's what I do.


Letter Grade System

What you think it says: "Our perceptive editorial staff recognizes that the true strength of the 100 point scale is how it relates back to your elementary school report card. Rather than muck about with numerical scores that can be interpreted, we're going to cut straight to the underlying metaphor, which can't be debated using mathematics."

What it secretly says: "Our editorial staff read Entertainment Weekly growing up, and therefore, have horrible taste and shouldn't be listened to."

Advantages:

- You've freed yourself from the 100 point scale, for the most part. Yeah, people can still do the math, but the meaning of a C+ is less debatable than the meaning of a 7.9.
- It's familiar to schoolchildren everywhere.
- You can work out your secret sadistic teacher fantasies, particularly if you write your reviews in red pen.

Disadvantages:

- Canada. Did I say a C+ was less debatable than a 7.9? Silly me. Turns out there are a lot of different interpretations of the letter grade scale depending on what region of the world you grew up in.
- Not to mention that your audience has wildly different views of the acceptability of certain scores, based primarily on how well they did in school. A straight A student reading your review will probably feel differently about a B minus than one of your readers who had to eek out a passing C. Also, what's a D in your personal interpretation? Still passing? Do you use that mysterious E?
- Makes your staff look like a pack of wannabe schoolmarms.

Kris's Recommendation: You can live with this system, but I wouldn't go around doing Billy Idol fistpumps in celebration. You're still reminding your readers of school in every single review, which can't have been the idea.


No Scoring System

What you think it says: "Our academic editorial staff facilitates a heuristic exploration of the textual/metatextual values vis-a-vis the cultural and aesthetic physiognomy of current electronic entertainment media."

What it secretly says: "We're assholes."

Advantages:

- You can probably convince your mom that you're not merely wasting your life writing video game reviews.

Disadvantages:

- Congratulations. You're the George Will of video games
- Readership figures in the tens of tens.

Kris's Recommendation: Obviously there's a place for serious academic studies of video games. It's also true that scores are generally overrated by gamers who read reviews. But the sort of writers who want to get rid of scores all together, yet still have hope that anyone is going to read their impenetrable prose, are not the sort of people you want to hang around with. It is possible to take video game reviews too seriously.

URL to share this content (right click and copy link)
Posted: 08/06/12, 00:13:47  - Edited by 
 on: 08/06/12, 23:42:39    
 
Why not sign up for a (free) account and create your own content?
 
Haha, classic updates, both of them! They should probably be added to the original post, even.


Posted by 
 on: 08/06/12, 23:28:38
I enjoyed your editorial and I agree with almost all of the points. I like the 100 point system even though I can't really explain what's the difference between a 7.3 and 7.4 besides I liked the 7.4 game more than the 7.3 one.


Posted by 
 on: 08/07/12, 00:13:19  - Edited by 
 on: 08/07/12, 00:14:10
@kriswright

I'm a Vulcan..


Posted by 
 on: 08/07/12, 00:33:30
I think 5 star is the way to go...

***** - everyone who owns this system is going to want this game.
**** - most people who own this system and everyone who likes this game's genre is going to want this game.
*** - about 50% of the people who own this system are going to like this game.
** - not very many people are going to like this game.
* - pretty much no one is going to like this game.


Posted by 
 on: 08/07/12, 00:45:44
@chrisguy

I don't think scores should reflect how many people the reviewer thinks will like the game. That's how GI ended up giving Paper Mario 2 a crappy score.

You also end up giving every CoD game 5 stars and every niche game two stars.


Posted by 
 on: 08/07/12, 00:55:31  - Edited by 
 on: 08/07/12, 00:56:23
Kris, I just noticed that your sense of humor is very Strong Bad-ian. I think that's why I like it so much.


Posted by 
 on: 08/07/12, 03:30:42
I just wanted to say that people are rarely lucky for being in the Twilight Zone. It's usually pretty bad news.


Posted by 
 on: 08/07/12, 03:46:03  - Edited by 
 on: 08/07/12, 03:46:22
@Anand That little boy will send you to the cornfield...


Posted by 
 on: 08/07/12, 04:16:20
@Secret_Tunnel

Holy Crap! That compliment makes wanna get crazy-go-nuts! ;)

@Anand

Well, compared to writing for a site that uses the 20 point scale, I'd say it's a toss up.

According to Serling's opening narration to the show, The Twilight Zone is the realm of Imagination, no? So, if you're lucky, you're simply imagining things. Chilling things. Things like a 20 point scale. That was my point.


Posted by 
 on: 08/07/12, 08:36:38
kriswright said:

Disadvantages:

- Canada.

I think that pretty much says it all, don't you?


Posted by 
 on: 08/07/12, 09:46:39
Finally, a thread about review scores that I can wholeheartedly endorse.


Posted by 
 on: 08/07/12, 09:57:15
I think this belongs here...



Posted by 
 on: 08/22/12, 15:29:09
That might be true for online aggregates but not for individual reviewer scores. 3 stars should be an average score.

EDIT: Also, even though it should be obvious, I have to point out that's a 10 point scale.



Posted by 
 on: 08/22/12, 15:46:29  - Edited by 
 on: 08/22/12, 17:22:36
Yeah but for all practical purposes "average" might as well be "crap" for most people. Why would anyone bother with something "average" when there are so many better choices? Most wouldn't, unless it is some specific style / genre /etc. that really appeals to them and doesn't have much representation in the industry as a whole.

I mean, the type of people who give credence to reviews at all. Obviously lots of people play average / crap games but they're probably not led to them by reviews as much.

To be honest I feel, especially nowadays, like my time is too valuable to struggle through a game I'm only kind of enjoying. I can handle that for movies because they are a one shot deal, but games require such a big commitment...


Posted by 
 on: 08/22/12, 21:31:52
Well, if you don't want to watch or play something that's merely average, that's your call as a review reader. But that doesn't make average suddenly synonymous with crap, just because you've personally elected to not have time for merely average things.

I mean, if I were working on a 5 star system, I'd probably give NSMB2 3 stars, right now. But if I were working in Roger Ebert thumbs, it'd still be a thumbs up. Along with the written review (which is more important, of course) a reader who trusted my opinion should be able to figure out if they wanted to purchase the game or not. It wouldn't be my responsibility to inflate the score for people who only trust 4 or 5 star reviews.

Hell, in a 2 star review, while bad, isn't necessarily a mark that something is total Ninjabread Man crap. It'd have to have some significant flaws, but it'd still have its virtues, too. I'd give Other M a 2 star review - a thumbs down - but I don't hate the game. I don't think it's crap.

Besides, if you look at what SL posted closely, it's basically a 3 or 4 star scale, which is even more limited than the one I suggest is superior.


Posted by 
 on: 08/22/12, 22:03:36
@kriswright

Yeah I know it's an aggregate scale. Just thought it was amusing .


Posted by 
 on: 08/22/12, 23:28:22
@Shadowlink

HAHA!! I love that!

--Converting my 6.4, I can't believe that SFA would be 3 stars. Seems super high!


Posted by 
 on: 08/22/12, 23:36:23
I would consider a 6.4 roughly a 2 star review, personally. The problem is the 100 point scale suffers from serious grade inflation, so if you're working backwards from it to the 5 star scale, you have to translate the "meaning" of the score, rather than simply try to do the math. A 6.4 is a poor review in most people's view. So 2 stars it is.

That'd be my approach.


Posted by 
 on: 08/22/12, 23:46:54
With the Review Poll contest well underway, I thought this thread needed a well deserved Bump.

This article still makes me laugh. Yay Kris.
A point to mention though - the voting in the polls thread is clearly determined by how well the review was put together/written. The score at the end is determined by the individuals own personal scoring system - I'm curious as to what the finalists here all use to get their score.

For instance, Stache uses a 100 point scale but he really draws straws then throws a calculator on the floor and see what number comes up.


S


Posted by 
 on: 02/01/13, 20:08:40
Mr_Mustache said:
--Converting my 6.4, I can't believe that SFA would be 3 stars. Seems super high!

Wait, do you mean Street Fighter Alpha?!? Dude that game was awesome! Still quite playable today, too. Hope it comes to (Arcade) Virtual Console soon!

You just hate fighting games. For shame, sir!


Posted by 
 on: 02/01/13, 23:26:22
Browse    1  2  3