A Review Scoring System That Would Work

 Pages 1 2 3 NEXT
 

A Review Scoring System That Would Work

Review scores are ridiculous, but if we must have them, then here is a system that should be implemented if you don't want to be bothered reading the review.

Read Full Article

Nice read. Recommended. 7/10...

In all seriousness though, reviews of artistic things are always tricky since everyone will have different tastes. Like he said, numbers are easy to calculate function and things, but certain people may get more emotion out of a piece of artwork than others...

Silly you, you almost had me thinking this was serious.

Problem is, not everyone follows gaming news and critique. Some just see a title that attracts their attention and want a simple Yay or Nay to justify their spending. Of course, one obvious trap is that reviwers might rate poorly genres they dislike, tanking a game that might have been alright to a savier fan.

You won't see AAA studies pushing for the ban of scores either, seeying as scores make it easy to quantify success, aside from taking less space at the box's cover.

The purpose of the review score is to tell us whether to bother to read the review for a game we weren't interested in.

9-10: Wow, guess I should read the review to figure out what's so great about this. Maybe I'm interested now.
1-3: Ooo, guess I should read the review to enjoy the train wreck.
4-8: I'm not reading the cunting review.

Review sites could achieve the same effect by going with better titles or ledes. '[Game]: The [Site] Review' doesn't really do it.

I would rate this review of review-scores and reviewing methods as Essential.
At least I would if the ending conclusion didn't score a 0% on Functionality.

I considered reading this article, but it didn't have a recommended trophy.

I don't mind multi-faceted reviews that include multiple elements of the game (graphics, plot, etc.), nor do I mind a summary of pros and cons, but honestly, who are these people dropping $60 on a game they are unwilling to read a couple of thousand words on first?

It's really not that hard to read the worst metacritic score, the best one from a site you've heard of, and maybe one more from a site you like.

I don't see the problem with review scores. They're just very rough approximations of how positive the reviewer feels toward whatever they're reviewing.

It's only ever an issue when people expect the system to be perfect. I can give a book a 4 out of 5 because I think it's really good but not the most amazing thing ever. My thought out opinion is always going to be more useful, but 4/5 is a nice way to put those thoughts in context.

Hmm I don't know about this one.
On one hand it is obvious that an artistic construct of any kind is generally hard to reduce to a number. It is an abstract approximation of a subjective appreciation, we all know that. Unless as mentioned, you start rating the pure technical performance which doesn't really hold much meaning either.

That said, I don't think dismissing scores is the way to go. While I applaud outputs or people deciding to go fully sans score (no, not you Eurogamer), I think that as a whole, scores are useful. I know everyone is dismissive and rather judgmental of sites like Metacritic or Gamerankings (or rottentomatoes), but to me they do represent an amalgamation of the global subjective abstract appreciations of a certain game.
And this amalgamation is a good way to measure the pulse,it is a good way to see the extremes, and a good way to observe what people find positive and negative.

Now personally, I care very little for scores, if a game get's 7 or a 9 it is unlikely to really modify my level of interest. I mostly follow specific reviewers, and read what they have to say because I find their observations insightful, or erotic, or whatever, but even then I'll often disagree with them. And this more direct interaction that I can have with a handful of people, doesn't really clash with the numeric values, but instead they complement each other.

After all this age we're in is all about discerning what information we find valuable in the chaotic ocean of stuff that is out there, shouting. And numbers are a good way to sift though it all.

PS: I never thought as Yahtzee as a reviewer, but more a commentator... Not sure if that's just a distinction without a difference.

Perfect deployment of the word "cunting".

I rate it 10/10, present it with an Essential badge and award it a Purple Teddy Bear of Excellence.

BrotherRool:
I don't see the problem with review scores. They're just very rough approximations of how positive the reviewer feels toward whatever they're reviewing.

It's only ever an issue when people expect the system to be perfect. I can give a book a 4 out of 5 because I think it's really good but not the most amazing thing ever. My thought out opinion is always going to be more useful, but 4/5 is a nice way to put those thoughts in context.

I rather firmly believe that the problem is the readers. Yahtzee touched on it a little, but it's the people who come into a review just to reaffirm their preconceived notions; when they see a number they disagree with for whatever reason, they have to throw a massive tantrum all over the internet because apparently other people holding different opinions is just something they cannot reconcile with their world view.

Of course, that point of view comes with the issue of not having an easy fix re: "get rid of review scores" and/or "read the review", because we would be trying to change the way people themselves actually act. And people generally seem to be fairly stubborn and resistant to change.

shrekfan246:

BrotherRool:
I don't see the problem with review scores. They're just very rough approximations of how positive the reviewer feels toward whatever they're reviewing.

It's only ever an issue when people expect the system to be perfect. I can give a book a 4 out of 5 because I think it's really good but not the most amazing thing ever. My thought out opinion is always going to be more useful, but 4/5 is a nice way to put those thoughts in context.

I rather firmly believe that the problem is the readers. Yahtzee touched on it a little, but it's the people who come into a review just to reaffirm their preconceived notions; when they see a number they disagree with for whatever reason, they have to throw a massive tantrum all over the internet because apparently other people holding different opinions is just something they cannot reconcile with their world view.

Of course, that point of view comes with the issue of not having an easy fix re: "get rid of review scores" and/or "read the review", because we would be trying to change the way people themselves actually act. And people generally seem to be fairly stubborn and resistant to change.

Stupid people. They always get in the way of an efficient society.

In terms of gameplay functionality, we also keep a tally of every time the player lost immersion for whatever reason, be it a glitch or poorly-written piece of dialogue, as well as one for each time they reached a failure state due to something that wasn't their fault, as determined by a brief inquiry from an unbiased panel of experts.

I wonder where something like Sunset Overdrive would fit on that scale? Breaking the 4th wall is the definition of losing immersion, but if done well it can be enjoyable and shouldn't be punished.

Also, I think Eurogamer's idea could work if they would just add an audience after each trophy. For example:
Call Of Duty: The Next One
Essential: Fans of the franchise, competitive FPS players
Recommended: People who enjoy tight mechanics in linear shooters with huge setpieces
Avoid: People who hate twitch gameplay, people who demand fresh and intricate stories

I understand and appreciate the sentiment behind this article. But I think Yahtzee may be missing the point entirely.

That being that any, and I mean any evaluation of a piece of art, and especially video games, is going to not only be a matter of personal opinion, but will also take more than a few sentences to fully explain, and the result of this unavoidable fact of life being that not everyone wants or needs your entire breakdown of each major aspect of the game, and just wants to make things faster and easier with an approximate summary of a reviewer's opinion represented in a single bit. This is why Metacritic exists, because being able to get at least a vague idea of a person's recommendation (or condemnation) and integrating it within the structure of several other such graph points can give you an idea of whether a game will be enjoyable or not in a much faster way than having to read half a dozen full reviews.

The truth is, we need both approaches to game reviewing.

I always translated scores into grades from school.
6 or below is failing, thus not worth buying.
7 is barely passing, but decent. Wait for a bundle release with other 7s.
8 is good, but not worth full price. Wait for a discount.
9 is amazing and worth full price, but no extras and debatable if worthy of getting at launch.
10 is PERFECT. As in flawless. Pre-order it, get the DLCs, get it at launch, have its children, play the living hell out of it forever and ever.

As a side note, no game has ever been perfect, no matter what marketing departments pay for.

I think the best way to understand the review score is to understand the review. Any number system is going to have its flaws, but most notably how a differently number scores are given. Some may give out 9/10 scores like candy while others very rarely give a 9/10. Its not because the numbers are inherently wrong, its because the reviewers have a separate litmus. Nintendo Power used to have a list of all the reviewers at the end of the review section, ranking their genre preferences from most preferred to least preferred. Were there flaws in that? Hell yes, but at the same time, you were given an insight into the reviewers personal preferances. The best way to understand the score is to understand the reviewer. Unfortunately, we don't have the ability to be super familiar with all the reviewers, so any attempt to score a game is going to have that flaw of miscommunication, and even if we do know them, even know them well, we will never understand them completely - there will never be a review where you and the reviewer are on the exact same wavelength, period. Doesn't mean we shouldn't try to mitigate it, however. I have my own confusing, poorly-thought out metho below.

A complicated mess that takes time, but its thorough and actually gives you some insight into the reviewers personal tastes. I'm sure that there's plenty of ways that it could be improved. Its about being pragmatic, not perfect.

Thunderous Cacophony:

shrekfan246:

BrotherRool:
I don't see the problem with review scores. They're just very rough approximations of how positive the reviewer feels toward whatever they're reviewing.

It's only ever an issue when people expect the system to be perfect. I can give a book a 4 out of 5 because I think it's really good but not the most amazing thing ever. My thought out opinion is always going to be more useful, but 4/5 is a nice way to put those thoughts in context.

I rather firmly believe that the problem is the readers. Yahtzee touched on it a little, but it's the people who come into a review just to reaffirm their preconceived notions; when they see a number they disagree with for whatever reason, they have to throw a massive tantrum all over the internet because apparently other people holding different opinions is just something they cannot reconcile with their world view.

Of course, that point of view comes with the issue of not having an easy fix re: "get rid of review scores" and/or "read the review", because we would be trying to change the way people themselves actually act. And people generally seem to be fairly stubborn and resistant to change.

Stupid people. They always get in the way of an efficient society.

Pretty much.

I know that I upgraded myself to the HuMan CPU when it came out, so that I would no longer need to be burdened by petty things like emotion. Unfortunately, it would seem that not many followed in my path.

"All it takes is for a 'Recommended' or an 'Essential' to be given to one game that's isn't all that, and the title is meaningless." - I had to read this sentence several times and have multiple massive brain hemorrhages before I realised there was an extra s in there tripping me up. Well played Yahtzee.

I'm not sure why some people hate scores so much. Scores are valuable. Scores are data. Data beats parsing language every time. And the people who say "You can't rate art!" annoy me. Yes you can. Just give it a score out of 100. It's easy to rate things.

Most reviews are meaningless rubbish that parrot the basic information about a game but rarely tell me what I need to know. Most professional reviewers are too caught up in telling us what they thought about the game rather than doing their jobs and trying to tell me whether *I* will like the game or not. Too many reviewers have become arrogant to think their opinion somehow matters more than everyone else's, especially the target audience for specific games.

So usually I read a few of the reviews from the top, bottom, and middle of the score spectrum. Give me 50 reviews with no scores and I'll just ignore them all.

That said, if video games wanted to go with a rottentomatoes approach with an aggregated pass/fail to produce a quality score I'd be right behind that.

shrekfan246:

BrotherRool:
I don't see the problem with review scores. They're just very rough approximations of how positive the reviewer feels toward whatever they're reviewing.

It's only ever an issue when people expect the system to be perfect. I can give a book a 4 out of 5 because I think it's really good but not the most amazing thing ever. My thought out opinion is always going to be more useful, but 4/5 is a nice way to put those thoughts in context.

I rather firmly believe that the problem is the readers. Yahtzee touched on it a little, but it's the people who come into a review just to reaffirm their preconceived notions; when they see a number they disagree with for whatever reason, they have to throw a massive tantrum all over the internet because apparently other people holding different opinions is just something they cannot reconcile with their world view.

Of course, that point of view comes with the issue of not having an easy fix re: "get rid of review scores" and/or "read the review", because we would be trying to change the way people themselves actually act. And people generally seem to be fairly stubborn and resistant to change.

Too true, apparently when sites remove review scores there's often a noticeable drop off in traffic to their reviews. And if they're willing to stop looking at a review altogether because of changes to the score, then the chances of convincing people to treat review scores in a more sane and less obsessive way... they're not high.

I think it's unfair to call review scores "meaningless", they clearly have some meaning, even if it is fairly ambiguous and reductionist. But people use reductionism all the time. When my parents ask me how my day went I usually utter a one or two word reply like "good" or "nothing special" and typically that's good enough. They don't need a complete rundown of my entire day and a detailed analysis of everything that happened.

Basically there are many degrees of detail that one can go into with an evaluation. A review score is the low-end extreme, a longform review is the other, neither is inherently superior.

I agree that single scores are largely meaningless and unhelpful. I am not convinced they are completely without merit, however. The only time I find they are helpful is if they are very low, especially given 7/10 is the minimum most sites will usually give a game, since it suggests there are some fairly objective reasons the game would not be worth my time rather than relatively subjective ones such as the reviewer not being a fan of the genre.

I think multi score systems could be a bit better, but as Yahtzee said for one to be 'accurate' it would end up being more cumbersome than reading a review. The best, simple version I came up with would be Technical, Story, and Gameplay. Technical would be the presence of bugs and crashes, as well as the performance of the game's graphics. Unfortunately, as mentioned by Yahtzee, to make this category really accurate you would need an entire QA department so my system is flawed from the very start. Story would consist of judging the quality of the characters, plot, dialogue, etc as I think would be obvious from the name. This category could possibly be the most subjective. Gameplay would be judging the quality and/or uniqueness of the game's mechanics and how enjoyable the parts of the game where story is not happening are. Also a fairly subjective category and could vary greatly depending on the style of play. Certainly not a perfect system, most notably because it is somewhat tailored to narrative driven games, but I think it is quite a bit better than a context free single score.

Another issue is whether or not reviews themselves are becoming obsolete with the prevalence of Let's Plays that allow potential buyers to see some actual gameplay and decide whether or not it appeals to them based on their own preferences, rather than those of a reviewer.

Scores are fine.

People operate on like-o-meters. You just know if you like product X more than Y and Y more than Z. Not difficult at all.

The only thing left then, before you can quickly start scrapping games off the long list is to have a hunch about what makes the critic tick. Maybe look for a decent critic or two, or just make do with the boring average of tastes on metacritic.

Reading reviews is for when your interest in a game is piqued.
Let's plays are for when you're either really on the fence about a game, or just really bored and you also don't mind massive spoilers.

Scores are the quick way to a manageable list.

I agree with the other people in this thread. Scores are fine. After all, we can't expect people to understand there are nuances and subtleties that might affect people's gameplay experiences in a way that's not easily categorized into a numerical ranking system. And that small details are just as important as large ones if they affect people in big ways.

So yes. Let's keep a scoring system. Because if I want to disagree with a criticism/review, I want to be able to do it by dismissing an arbitrary number, not actually engaging the criticisms.

shrekfan246:
I rather firmly believe that the problem is the readers. Yahtzee touched on it a little, but it's the people who come into a review just to reaffirm their preconceived notions; when they see a number they disagree with for whatever reason, they have to throw a massive tantrum all over the internet because apparently other people holding different opinions is just something they cannot reconcile with their world view.

This is pretty much why everyone reads reviews though.

Hell, we see people getting mad over reviews that fail to confirm their feelings on games they haven't even played yet.

The only time people read reviews for actual information or opinions is when the review is of a game they haven't heard of and they're reading it out of curiosity.

shrekfan246:

BrotherRool:
I don't see the problem with review scores. They're just very rough approximations of how positive the reviewer feels toward whatever they're reviewing.

It's only ever an issue when people expect the system to be perfect. I can give a book a 4 out of 5 because I think it's really good but not the most amazing thing ever. My thought out opinion is always going to be more useful, but 4/5 is a nice way to put those thoughts in context.

I rather firmly believe that the problem is the readers. Yahtzee touched on it a little, but it's the people who come into a review just to reaffirm their preconceived notions; when they see a number they disagree with for whatever reason, they have to throw a massive tantrum all over the internet because apparently other people holding different opinions is just something they cannot reconcile with their world view.

Of course, that point of view comes with the issue of not having an easy fix re: "get rid of review scores" and/or "read the review", because we would be trying to change the way people themselves actually act. And people generally seem to be fairly stubborn and resistant to change.

The assumption that people only use review scores just as internet flame ammo and to confirm their preconceived notions is a very shaky assumption, often asserted but never backed up by any evidence. Frankly, you stating that assumption says more about what you think of people than how people actually are.

In other words, just because you are a cynical bastard doesn't mean everyone else is an idiot.

http://xkcd.com/610/

DrOswald:

shrekfan246:

BrotherRool:
I don't see the problem with review scores. They're just very rough approximations of how positive the reviewer feels toward whatever they're reviewing.

It's only ever an issue when people expect the system to be perfect. I can give a book a 4 out of 5 because I think it's really good but not the most amazing thing ever. My thought out opinion is always going to be more useful, but 4/5 is a nice way to put those thoughts in context.

I rather firmly believe that the problem is the readers. Yahtzee touched on it a little, but it's the people who come into a review just to reaffirm their preconceived notions; when they see a number they disagree with for whatever reason, they have to throw a massive tantrum all over the internet because apparently other people holding different opinions is just something they cannot reconcile with their world view.

Of course, that point of view comes with the issue of not having an easy fix re: "get rid of review scores" and/or "read the review", because we would be trying to change the way people themselves actually act. And people generally seem to be fairly stubborn and resistant to change.

The assumption that people only use review scores just as internet flame ammo and to confirm their preconceived notions is a very shaky assumption, often asserted but never backed up by any evidence. Frankly, you stating that assumption says more about what you think of people than how people actually are.

In other words, just because you are a cynical bastard doesn't mean everyone else is an idiot.

http://xkcd.com/610/

Yes, I have a very low opinion of the people who are often the loudest within the gaming community. I am very comfortable admitting to that.

However, I did not say that people only read reviews to strengthen positions they already held; I said that the people who only read reviews to throw tantrums (or, alternatively, to smugly scoff about how right they were about how good/bad something is) are the ones I view to be the big problem with reference to review scores.

EDIT: Edited for a word choice.

I simply dont measure games on anything past:
-Optimization
-Stability
-technical competence (graphical fidelity, sound quality, AI, physics)
-Things like options menus, color blind modes, MODDING as a possibility.

That is objective.
Everything past that is subjective territory.

I mean people liked the bad mess of a game that was Bioshock Infinite... which is technologically competent I must admit.

Subjectivity is how I keep myself sane :P

Zhukov:

shrekfan246:
I rather firmly believe that the problem is the readers. Yahtzee touched on it a little, but it's the people who come into a review just to reaffirm their preconceived notions; when they see a number they disagree with for whatever reason, they have to throw a massive tantrum all over the internet because apparently other people holding different opinions is just something they cannot reconcile with their world view.

This is pretty much why everyone reads reviews though.

Hell, we see people getting mad over reviews that fail to confirm their feelings on games they haven't even played yet.

The only time people read reviews for actual information or opinions is when the review is of a game they haven't heard of and they're reading it out of curiosity.

I can't speak for other people, but I read reviews to get information even if it's a game I've been following.

More often I watch videos of the game in question, admittedly, as there are few reviewers or critics I follow who regularly put out written content, but I even read Jim Sterling's review of Final Fantasy X HD before I bought it, despite it being a re-release of a game I've owned for over a decade.

Honestly, sometimes I really do feel like I'm some kind of robot when it comes to the area of review and critique in video games, because I just can't fathom how incensed people get over these subjects. It makes no sense to me.

Silentpony:
I always translated scores into grades from school.
6 or below is failing, thus not worth buying.
7 is barely passing, but decent. Wait for a bundle release with other 7s.
8 is good, but not worth full price. Wait for a discount.
9 is amazing and worth full price, but no extras and debatable if worthy of getting at launch.
10 is PERFECT. As in flawless. Pre-order it, get the DLCs, get it at launch, have its children, play the living hell out of it forever and ever.

As a side note, no game has ever been perfect, no matter what marketing departments pay for.

That's kinda of a problem because the reviewers are the ones who decide what their numbers represent. And as there is no official standard on that matter, lots of conflicts and misinterpretations appear. For example, this is the IGN's version (rounded down):

0 - disaster
1 - unbearable
2 - painful
3 - awful
4 - bad
5 - mediocre
6 - okay
7 - good
8 - great
9 - amazing
10 - masterpiece

Now, their most famous score is the joke of the Internet (Pokemon ORAS: 7.8 - too much water), even if their score means "good".

Someone once stated that the gaming community treat the 7/10 games the same way as the movie critics treat the 2/5 stars movies. The difference is that lots of 2/5 stars movies are really popular with the general public (or they just don't care about the ratings), and that isn't the case in gaming.

Over the years I have taken the approach, there is the review commentary and then there is the review score. I read the review because I want to know what the reviewer thought about the game and their experience with it.

When the Gamespot mess happened and a reviewer was fired over a review score he gave, I started seeing a disconnect between what was said and what score was given. Pretty soon most games received a 7 or higher or the magical 8.5 Generally I find that if the game came from a publisher that did not supply advertising $$$ then maybe the score is more reflective of the actual review.

So any kind of stars, marks, chits, thumbs up etc is fine provided that the real review is still written in the review itself. We can leave the rating wars to those that are too busy storming the castle over number - raaahhh 7.3!?! how dare you sir! Have at thee!

irishda:
I agree with the other people in this thread. Scores are fine. After all, we can't expect people to understand there are nuances and subtleties that might affect people's gameplay experiences in a way that's not easily categorized into a numerical ranking system. And that small details are just as important as large ones if they affect people in big ways.

So yes. Let's keep a scoring system. Because if I want to disagree with a criticism/review, I want to be able to do it by dismissing an arbitrary number, not actually engaging the criticisms.

I can't tell if that last sentence is a sad truth or pure cynicism...

I recall several old gaming magazines that had subscores for their reviews. Not detailed enough to cover what yahtzee is suggesting, but it did remind me of them...

They had an overall score and a breakdown into 4 or 5 areas for things like graphics, gameplay, sound and story.

While still nowhere near detailed enough to give you any truly meaningful information, it did mean you could, at a glance get an idea of say:

Game 1: Really shitty graphics, but fun gameplay.
Game 2: Very, very pretty, but has a terrible story, and is really quite dull to play (or outright broken)

Not that it really helps much, but it was interesting...

I think a 10 point scale just has this predisposition to become skewed. A 5 just inherently looks like shit to a lot of people.
I've seen this in other places where such a grading scale has been used.
It was almost always scores between 7 and 9 and when it dipped lower to 5 or 6 people around me reacted to how low it was.
Anything below 5 was pretty much never used.

I think it might just be something we just naturally do. I'm probably wrong though.

I still feel all these years later that the best numerical review system (and the one that truly worked) was the old EGM review system. Yes it was a standard 10 point scale with half points in between, but it included written reviews by 4 different reviewers. Each reviewer provided their own score and their own reasons for liking/disliking the game as well as having the opportunity to counter comments made by the other reviewers. This sort of approach doesn't really fix the problems found in a numerical (or award) based review system, but it very clearly displays that games are experienced differently by different people with different tastes. In EGM, iIf a game got really high scores by every reviewer you knew you were likely looking at a very high quality title.

I feel like review scores get even more messy in complicated because of the response to them. Some people go by basic logic of how numbers work (i.e. 5 is acceptable as it is in the middle), whereas some others will go by the education logic of 7 being the bare minimum which is insane to somebody like me who sits in the former category.

I do like looking at a review score just fine, but I don't see how they add to the actual text so I very much agree. Seeing Jim Sterling give the 8/10 to ZombiU after reading his review just gave me a very brief "aww yis" feeling, nothing more.

The only reason review scores can be bad is because people treat them as if they have to be an objective unchanging definitive way of ranking every single game in comparison to every game. If you are only using the score as a judge of whether you might like the game or not then you are an idiot.

Any review is likely to contain negative and positive aspects of a game and it's sometimes nice to have a score as part of the summary. Not an Essential aspect, nor is it Recommended or should it be Avoided. It's just an option.

 Pages 1 2 3 NEXT

Reply to Thread

Log in or Register to Comment
Have an account? Login below:
With Facebook:Login With Facebook
or
Username:  
Password:  
  
Not registered? To sign up for an account with The Escapist:
Register With Facebook
Register With Facebook
or
Register for a free account here