This started as a post in the "IGN switching to a 20 point scale..." thread, but then my usual typing diarrhea started up and I decided to put it out as an editorial. There are a lot of scales competing for use amongst video game sites. This is just one man's opinion on them.


100 Point Scale

What you think it says: "Our sophisticated editorial staff can evaluate, with laser-like accuracy, the success or failure of a particular game."

What it secretly says: "Our editorial staff are fulfilling their childhood dreams of writing for late 80s/early 90s video game magazines. Particularly if this 100 point scale comes coupled with a box score that rates the sound and graphics separately. And double that if it comes with funny cartoon pictures."

Advantages:

- An individual reviewer could use this scale to rate their game preferences with pinpoint accuracy.
- It's an institution. The 100 point scale, based out of 10, with decimals, has been the default setting in gaming rags for so long that most everyone understands it, despite any obvious flaws.
- You can give a game a 9.9 and still think of yourself as being fair and balanced. "Hey, I didn't say it was perfect!"

Disadvantages:

- There's no way an editorial staff can agree on anything with pinpoint accuracy, so that whole premise is flawed.
- Even an individual reviewer is going to have a hard time telling you the difference between, say a 7.3 and a 7.4. At that point of precision, isn't the reviewer's mood that day just as important as anything in the game?
- Like it or not, everything below an 8 is going to be considered a failure by a significant chunk of your audience, even though that makes no sense.
- Your audience constantly debates whether the scale works or not. Have fun listening to that. It ain't going away.

Kris's Recommendation: Stick with this if you're already using it and it's working. Yeah, it's a charade and yeah hearing people complain about it is a pain in the ass, but chances are you're already a big box gaming site, anyway. If you have problems with your readership figures, it's probably not because you have an illogical rating scale.


20 Point Scale

What you think it says: "Our reasonable editorial staff knows that there are flaws with the 100 point scale, so we've come up with an original solution that everyone can agree on, one that gives us greater flexibility while still removing some of the more ridiculous characteristics of the 100 point scale."

What it secretly says: "Our unreasonable editorial staff can't agree on jack shit. No one's happy with this idiotic compromise, but it was the only way we could keep everyone from killing each other. Hope you like the number 8.5, because you're gonna be seeing a lot of it."

Advantages:

- Your editorial staff stops fighting and simply goes back to grumbling to themselves.
- The gambling pool for how your site will score a game just got a whole lot easier.
- 4 idiots in a small town in Nebraska think this is the best system ever. At least they're happy.

Disadvantages:

- 8 is still going to be the base level for success or failure with a significant portion of your audience. You've simply trimmed all the wiggle room above that.
- You still get to spend all your time insisting that a 7 is a pretty good score, even though no one believes you. In fact, with this new system, you get to have that argument more!
- Your entire staff looks like a pack of morons.

Kris's Recommendation: Start looking for a new job if everyone you work with agrees this is the best scale. Or pinch yourself. Hey, if you're lucky, you are in the Twilight Zone.


10 Point Scale

What you think it says: "Our pragmatic staff recognizes that larger scales are silly. We need plenty of room to rate games, but we're not going to try to snowball you into thinking we need 100 points of accuracy. We respect your intelligence."

What it secretly says: "Our sentimental staff just can't live in a world where Ocarina of Time doesn't have the number 10 next to it."

Advantages:

- You get flexibility without the illusion of precision.
- You can probably convince more people that a 7 is a pretty good score. I mean it's not like you can give every game an 8, 9 or 10.

Disadvantages:

- A significant portion of your audience still thinks you better give every game either an 8, 9 or 10.
- When the big box 100 point scale guys give a game an 8.5 and you give it a 7, you have to explain that those are similar scores, even though no one believes you. And you get to do this every time.

Kris's Recommendation: Well, you can sleep at night, anyway. But your scale is still too easily comparable to the 100 point scale you're probably trying to get away from. Plead that you've solved that problem all you want, but your logic doesn't work here.


5 Star System

What you think it says: "Our wise editorial staff understands that you look at ratings simply to determine if we recommend a game or not. You don't expect pinpoint accuracy. You just want the straight dope."

What it secretly says: "Our grumpy editorial staff hates rating games in the first place."

Advantages:

- FINALLY, there's a system that breaks free from comparisons to the silly 100 point scale!
- The system is easy to understand, particularly since it's been used for years by film critics in newspapers.
- Every point on your scale means something. You can give something 2 stars with no problem.

Disadvantages:

- You're going to be constantly tempted to shit out half-stars, which defeats the whole purpose of going to a 5 star system.
- You have somewhat sacrificed flexibility for having a logical system. Hope that helps when you can't decide whether to give a highly anticipated game a 4 or a 5.
- Half of your editorial staff will mourn for 6 months about Ocarina of Time not having the number 10 next to it. Some of them will try to kill you.

Kris's Recommendation: Consider yourself a hero. This is the most respectable scale to use, period. Catch me in a bar, I'll buy you a drink.


Up or Down

What you think it says: "I'm the Roger Ebert of video games."

What it secretly says: "I'm not the Roger Ebert of video games."

Advantages:

- Well, at least you cut to the chase.

Disadvantages:

- Look, unless you have the credibility in stock already, no one is going to listen to your Up or Down scale. Even Siskel and Ebert only made the system work by fighting with each other over it every week in defense of their views. And, even then, it was technically a 4 point scale, because the real endorsement was if the two them agreed and both gave the thumbs up.

Kris's Recommendation: Look at that guy in the mirror. See that guy? That guy needs some perspective on his place in the Universe. Maybe take up stargazing or bird watching. Resist the urge to rate the birds, though.

Or you can just yell at them. That's what I do.


Letter Grade System

What you think it says: "Our perceptive editorial staff recognizes that the true strength of the 100 point scale is how it relates back to your elementary school report card. Rather than muck about with numerical scores that can be interpreted, we're going to cut straight to the underlying metaphor, which can't be debated using mathematics."

What it secretly says: "Our editorial staff read Entertainment Weekly growing up, and therefore, have horrible taste and shouldn't be listened to."

Advantages:

- You've freed yourself from the 100 point scale, for the most part. Yeah, people can still do the math, but the meaning of a C+ is less debatable than the meaning of a 7.9.
- It's familiar to schoolchildren everywhere.
- You can work out your secret sadistic teacher fantasies, particularly if you write your reviews in red pen.

Disadvantages:

- Canada. Did I say a C+ was less debatable than a 7.9? Silly me. Turns out there are a lot of different interpretations of the letter grade scale depending on what region of the world you grew up in.
- Not to mention that your audience has wildly different views of the acceptability of certain scores, based primarily on how well they did in school. A straight A student reading your review will probably feel differently about a B minus than one of your readers who had to eek out a passing C. Also, what's a D in your personal interpretation? Still passing? Do you use that mysterious E?
- Makes your staff look like a pack of wannabe schoolmarms.

Kris's Recommendation: You can live with this system, but I wouldn't go around doing Billy Idol fistpumps in celebration. You're still reminding your readers of school in every single review, which can't have been the idea.


No Scoring System

What you think it says: "Our academic editorial staff facilitates a heuristic exploration of the textual/metatextual values vis-a-vis the cultural and aesthetic physiognomy of current electronic entertainment media."

What it secretly says: "We're assholes."

Advantages:

- You can probably convince your mom that you're not merely wasting your life writing video game reviews.

Disadvantages:

- Congratulations. You're the George Will of video games
- Readership figures in the tens of tens.

Kris's Recommendation: Obviously there's a place for serious academic studies of video games. It's also true that scores are generally overrated by gamers who read reviews. But the sort of writers who want to get rid of scores all together, yet still have hope that anyone is going to read their impenetrable prose, are not the sort of people you want to hang around with. It is possible to take video game reviews too seriously.

URL to share (right click and copy)

Wouldn't Siskel and Ebert's combined score be a 3 point scale, not a 4 point? Whether it was Siskel or Ebert who gave the thumbs down, it's still a thumbs up / thumbs down review. So you get 2 thumbs up, mixed, or two thumbs down. 3 points!

Posted by

Zero

on: 08/06/12, 00:27

@Zero

Well... I actually debated that. But I'm working under the assumption that most people had a preference for either Siskel or Ebert. So, you know, for some people it would be a 4 point scale with Siskel's individual endorsement carrying more weight than Ebert's or vice versa.

But, sure, it's debatable.

Posted by

kriswright

on: 08/06/12, 00:28

Gold! Don't ever stop writing Kris. (Maybe submit this to Cracked?)

But let's be fair here:

What it secretly says: "Our sentimental staff just can't live in a world where Ocarina of Time doesn't have the number 10 next to it."

A world such as the one you describe simply isn't worth living in. Search your feelings. You know it to be true.

Posted by

Shadowlink

on: 08/06/12, 00:38

@Shadowlink

7.9

Posted by

kriswright

on: 08/06/12, 00:42

Love it!

Posted by

Secret_Tunnel

on: 08/06/12, 01:04

brb printing out the OP and rubbing it all over myself


I use the 10-point scale in mine, but only because it was already set up at the site. 5 is the only correct choice.

Posted by

nate38

on: 08/06/12, 03:01

Most excellent, and I agree with all your points. Well written.

Posted by

ploot

on: 08/06/12, 03:02

nate38 said:

brb printing out the OP and rubbing it all over myself.

I thought I was the only one who did that!

My favorite site that folded back when the first .com bubble burst, Dailyradar, used a 4 point scale. Direct Hit, Hit, Miss and Dud. It was fantastic.

Posted by

Guillaume

on: 08/06/12, 03:22

Edited:

08/06/12, 03:28

@Secret_Tunnel
@nate38
@ploot
@Guillaume
Does that mean he's going to buy us all drinks?

Posted by

CPA Wei

on: 08/06/12, 03:42

Edited:

08/06/12, 03:42

@CPA Wei

Yep. Cement Mixers, all around.

Posted by

kriswright

on: 08/06/12, 06:08

I think this point from the 100-point scale part of your post rings true for each category:

"- Your audience constantly debates whether the scale works or not. Have fun listening to that. It ain't going away."

Posted by

GameDadGrant

on: 08/06/12, 15:42

Great Read!

I'm still trying to find something that everyone likes but I don't think that will be possible.... My personal reviews are out of 10 (although it's in .5 increments so I guess I fall in the 20 point scale) but mine is figured out mathematically. For instance, I don't rate the graphics, sound and play control each out of 10... it's a total, and it's weighted according to what I think is more important in a game. You can't get a great score out of pretty graphics and high quality Dolby Digital Surround Sound alone.

Gameplay / 3
Play Control / 3
Graphics and Sound / 2
Fun Factor / 2
Total /10

Yes, there are a few .5 scores but I find it easier then trying to do the .1 difference. I can't handle that and I have too many games that I think are better then a 7..but not quite an 8. There are a few games I've rated a 10 that some people might disagree with, but if everybody loved my system, then it would be perfect - and I don't think any rating system can be - for everyone.

S

Posted by

Smerd

on: 08/06/12, 17:46

Edited:

02/01/13, 20:09

I notice you didn't mention 1up's letter grade system which I think may be the best system for avoiding most of the trips and traps you discuss.

Posted by

nacthenud

on: 08/06/12, 18:50

The 5 point system is jocked in this thread. Kris and I are soulmates confirmed.

Posted by

Kal-El814

on: 08/06/12, 18:54

I like the 20-point scale myself.

I rate this thread 9.5 /10.

Posted by

anon_mastermind

on: 08/06/12, 19:26

Haha, I like this thread. But I would like to see letter scores edited in and similarly satirized.

Posted by

TriforceBun

on: 08/06/12, 19:38

Thanks for the good words, guys. I do think that there's a difference between how these systems work with individual reviewers, editorial staffs and aggregates. Something like the 5 star system obviously wouldn't work for an aggregate like Metacritic and something like the 20 point system can work for an individual reviewer (I'd still have problems with it as a reader, but basically a singular voice can set the "meaning" of a scale a heck of a lot easier than 10 writers with 10 different interpretations of it).

Obviously, this was mostly for fun. It might have been clear that I think the biggest issue is that break between an 8.0 and a 7.9 on the 100 point scale. We've basically been trained, over years of use and abuse of the 100 point scale, that an 8 is a good score and a 7.9 is a disappointment, even though logically they're only a point away from each other. That wrecks the rest of the scale, in my opinion, and causes collateral damage to all scales that can be easily compared to it. There are other problems with the 100 point scale but that's the big one.

I mean, technically this is just mathematics. All of these scales would be usable if we were Vulcans and could agree on a single logical interpretation of them. But in reality, there are strong, opposing, subjective interpretations of what different points on a scale ought to mean. That's why, say, a 20 point scale strikes me as a worse scale than a 100 pointer - you carry all the baggage of the 100 point scale over, but get none of the precision. Whereas a 10 point scale frees you of some of the baggage (assuming people know you're even on it and not simply using a 100 point scale), even though there's no strictly logical reason it should be better. I think it's just easier to make the case with your audience that sub-8 scores are still worthy.

One thing I do think people ought to ask is why video games ever had a 100 point scale as a standard in the first place. There are people in the other thread (sorry for the redundancy, Zero) who complain about the 5 star system not being precise enough, but it was used with no complaints in film, food and hotel reviews for decades before video game mags cranked out the 100 pointer. I suppose my question is why do video games need greater precision in their scores than any of these others?

Okay. A couple of request in here. I've got two more entries I want to write and I'll post 'em in a bit. Thanks for reading/debating.

Posted by

kriswright

on: 08/06/12, 21:11

@kriswright

I absolutely disagree that a 5 point scale wouldn't work for an aggregator site! Statistically significant differences on those scales drive the approval of drugs to market and are a foundation of many kinds of research.

The differences might look less "sexy" than on something like metacritic, but they would absolutely work. And in fact, I'd argue that if you could get a significant number of reviewers to review a significant variety of games, not only would it work, but it would work MUCH BETTER than what they do now.

Posted by

Kal-El814

on: 08/06/12, 21:39

Edited:

08/06/12, 21:41

Letter Grade System

What you think it says: "Our perceptive editorial staff recognizes that the true strength of the 100 point scale is how it relates back to your elementary school report card. Rather than muck about with numerical scores that can be interpreted, we're going to cut straight to the underlying metaphor, which can't be debated using mathematics."

What it secretly says: "Our editorial staff read Entertainment Weekly growing up, and therefore, have horrible taste and shouldn't be listened to."

Advantages:

- You've freed yourself from the 100 point scale, for the most part. Yeah, people can still do the math, but the meaning of a C+ is less debatable than the meaning of a 7.9.
- It's familiar to schoolchildren everywhere.
- You can work out your secret sadistic teacher fantasies, particularly if you write your reviews in red pen.

Disadvantages:

- Canada. Did I say a C+ was less debatable than a 7.9? Silly me. Turns out there are a lot of different interpretations of the letter grade scale depending on what region of the world you grew up in.
- Not to mention that your audience has wildly different views of the acceptability of certain scores, based primarily on how well they did in school. A straight A student reading your review will probably feel differently about a B minus than one of your readers who had to eek out a passing C. Also, what's a D in your personal interpretation? Still passing? Do you use that mysterious E?
- Makes your staff look like a pack of wannabe schoolmarms.

Kris's Recommendation: You can live with this system, but I wouldn't go around doing Billy Idol fistpumps in celebration. You're still reminding your readers of school in every single review, which can't have been the idea.



No Scoring System

What you think it says: "Our academic editorial staff facilitates a heuristic exploration of the textual/metatextual values vis-a-vis the cultural and aesthetic physiognomy of current electronic entertainment media."

What it secretly says: "We're assholes."

Advantages:

- You can probably convince your mom that you're not merely wasting your life writing video game reviews.

Disadvantages:

- Congratulations. You're the George Will of video games
- Readership figures in the tens of tens.

Kris's Recommendation: Obviously there's a place for serious academic studies of video games. It's also true that scores are generally overrated by gamers who read reviews. But the sort of writers who want to get rid of scores all together, yet still have hope that anyone is going to read their impenetrable prose, are not the sort of people you want to hang around with. It is possible to take video game reviews too seriously.

Posted by

kriswright

on: 08/06/12, 22:02

kriswright said:

Disadvantages:

- Readership figures in the tens of tens.

HAHAHAHAHAHA

Posted by

Kal-El814

on: 08/06/12, 22:09