Jimquisition: Review Scores Are Not Evil

 Pages PREV 1 2 3 4 5
 

I don't like scores myself because they don't make a damn but of scene. I tried to ask some reviewers 10 years ago what do the scores mean and I got something to the effect of they are an opinion brought into easy number form. But to me let's say a 1 to 10 score was made then wouldn't 5 me OK not my cup of tea but not bad. But it goes to saying 5 out of 10 might as well be shit on a waffle for breakfast.

If I would do score I'd have to break them down in technical and opinion.

I see much more complaining on the internet about how scores average around 7-8, and any game that got a 5 (which *should* be average) is seen as total crap, than how scores suck altogether. They're just weighted funny, giving them much lower resolution. Has any game ever gotten a 1? Ever?

This is something that makes me cringe every time I see it.

Honestly, if you want to complain about where the average is, fine, but stop saying it should be five because "that's the average" because it isn't. An "average" is not necessarily based on what the middle number is, that's the median you're talking about).

And frankly there's nothing wrong with a 7 being "average", either, provided you apply your scale consistently. The problem isn't that the scale is bad, it's that the scale isn't applied with any degree of consistency and that larger companies have been known to be able to buy better ratings. Address the REAL problem rather than creating a fictional one.

Actually I believe the reason most people see anything below a 7 as abysmal is because the normal grading system used in the education system. 50% and below is an F, where as 60-69% is a D, 70-79% is a C, 80-89% is a B, and an A is 90-100%. Converting that to a 10 point scale, we get 7-10 being desirable scores while 1-6 are the scores we wish to avoid. We are basically taught that is the way things should be.

To me game scores do not work like they seem to for some others. I dont think they can work for comparing two games as much as saying how much a game lived up to the potential of its concept. Though some concepts are so bad there is no winning.

KiloFox:
i like the new gloves. they match the suit and make you look spiffy.
you don't wear enough Red for the red gloves to work in the same way.

anyway. i'm so-so on review scored... i don't really like how an 8 is seen as BAD. an 8 is simply 80%. it's still a B grade, and even without taking BS school grades into account it means you did 80% of stuff great! (making a 50% just "average" or "Decent") the scores themselves arn't bad, but rather it's the way that some people VIEW them that's really bad.

I agree. If the scoring system made sense (0 awful 1-3 bad, 4-6 average, 7-9 good, 10 exceptional or something like that) then I wouldn't mind critics using that system. Take Duke Nukem Forever, on this very website the reviewer slammed the crap (no pun intended) out of it, then gave it a 7.

Review scores are just dumb universally. The belief that it'd be a good idea to measure subjective opinions with numbers is pants-on-head retarded, even more so when you factor in there isn't even a consensus of what each number means. You can ask anyone if they like a game for example, if you asked a normal person how they felt about portal they'll say they loved it, hated it, or somewhere in between, not 9 or 4 stars or 13 gophers because such statements are nonsensical. In order to talk in terms of scores you need to translate your feelings into a number and then the reader has to translate your number back to feelings, even worse is that they often do so incorrectly.

Also, review scores are dishonest, they represent subjective opinions as objective facts; 9/10 of a pizza is universal, 9/10 on a game review is not.

Best Ending Ever.

I think what some people like about review scores is how they can very precisely sum up you're feelings towards a game in a short span.

6/10
done.

You can't do that with words without being vague. Of course that doesn't make them any less limited and trivial. But if you just want to pour over a bunch of reviews for a game without spending 5 minutes on each one numbers can at least give you something.

I don't really understand why the scores need to be so exact like 1/10 or even on a 100 scale, when their is to much subjectivity in that number. The only thing that is matter that is good, ok or bad. The one example of that (the reason why I stopped reading Jim's reviews) was Witcher2 and Skyrim. U see Jim criticized Witcher for different reasons and voila in Skyrim in some of the same areas was doing MUCH MUCH worse but there didn't count. Both games have cons and pros, both games are good, which are better is more a question of tastes (I liked more Witcher than Skyrim), so how does justify Jim score of a 6 and a 10. An 8 and a 9 that would be subjective but 6 and a 10, sorry Jim but I expect from a reviewer to be as objective as possible.

Right, I keep seeing this and now I must point it out.

Aurora Firestorm:
I see much more complaining on the internet about how scores average around 7-8, and any game that got a 5 (which *should* be average) is seen as total crap, than how scores suck altogether. They're just weighted funny, giving them much lower resolution. Has any game ever gotten a 1? Ever?

What do you mean by "Should be average". What on earth does ANYONE mean by that.

If the average game gets a 7 then 7 = Average.

The problem is that the average game gets a 7.

So unless a site specifically states that they use a different system from other sites etc et etc, 7 = Average.

There is no "Should" with averages.

Most people who hate review scores seem to forget that. Yes, it fucks up the system, but we all have internalized it. If something gets a 6 we dont think "This is an above average game" we think, quite rightly, "This game is below average". Because it is below the average that has already been set.

You see where I am going with this?

I have nothing against review scores. I believe them helpful. Nor do I have anything against either metacritic, well the review scores from professional critics at least.

I guess I hold a hated view these days. Yes, I accept that the system is a bit broken. But that aint the systems fault.

kodra:

I'm sensitive about this because of The Secret World, a game that was probably the most impressive steps forward in MMO content design in years, that was essentially killed a month after launch because it got a good number of middling review scores followed by one guy who basically torched the game based on his experience with the beta client.

What this tells me is that gamers aren't ready for games to innovate, because they'd rather play 95 metacritic score games than any that are willing to take any risks in upsetting the review-class journalists.

I don't mean to rain in your parade, but that is actually a really common misconception of today.. (and of a lot of people in history) Innovation, is a risky thing, it means to make NEW things (that is questionable on itself), but it doesn't necessarily make GOOD things. The fact that something is innovative, doesn't make it automatically GREAT! just NEW, but it very well can be HORRIBLE (which incidentally was my personal experience with the secret world, because let's face it, other than content design, the game was stagnant in EVERY OTHER TECHNICAL SENSE).

So, a bit off topic but lets not continue this fallacy!
Remember:

NEW != GOOD

GOOD != NEW :)

BETTER == NEW o

And on topic: people, people, stop saying 5 should be the average, 5 is the average out of 10, which does not mean that most games should be in that average. Please think about it, most games are rather decent, at least functionally, as a general law I'd say in fact that most games are in fact good, SO THE AVERAGE IS ABOVE THE MATHEMATICAL CENTER POINT. Tough luck that too many rather good games come out so often, since most people simply have to discriminate to their prefered genres/highest critic scores...

Why does every other Jimquisition contain images of poop? It doesn't go well with the fascist theme!

Doesn't Jim care about what the OSS wrote about that German dictator?

Cool gloves though. Please bring back the tie. Or perhaps a foulard would be nice.

vezon:
I don't really understand why the scores need to be so exact like 1/10 or even on a 100 scale, when their is to much subjectivity in that number. The only thing that is matter that is good, ok or bad. The one example of that (the reason why I stopped reading Jim's reviews) was Witcher2 and Skyrim. U see Jim criticized Witcher for different reasons and voila in Skyrim in some of the same areas was doing MUCH MUCH worse but there didn't count. Both games have cons and pros, both games are good, which are better is more a question of tastes (I liked more Witcher than Skyrim), so how does justify Jim score of a 6 and a 10. An 8 and a 9 that would be subjective but 6 and a 10, sorry Jim but I expect from a reviewer to be as objective as possible.

You've hit on why I don't follow review scores. I've read reviews where aspects (like shoddy cameras and repetitive levels) lowers a game to a 7 but another game as has the same issues and its still a 9.5.

Also reviews on multiplats are totally skewed. Just because you reviewed the 360 version of a game doesn't mean the PS3 version deserves the same score. There is no way that Skyrim for PS3 deserves that 10 (or 5) that it got.

Personally I think we should get rid of the number and just have the rankings be: Buy it Now, Buy it Later, Rent it, Never Look in this Games Direction.

I have always been a fan of the simple -/5 score system. It's very simple and gives you an idea of what the review is going to read like, without being ridiculously overstated. -/10 scores tend to exaggerated by quite a bit. Maybe it's because I was raised on GamePro, I dunno.

In example:

1 - Horrible, shoddy game that should never have existed. Technical errors, game breaking glitches, and poor graphics are just the start of where this went wrong.

2 - A game that is so bad that it is unplayable or not enjoyable. Games in this category usually have poor graphics, bad voice acting, a terrible story, or overly ambitious mechanics, and may also suffer from bad bugs. Should generally be avoided.

3 - A game. A mediocre title that is neither good nor bad, the game hits the mark of 'playable' and can be fun to play, but does nothing special to earn any recommendations. Usually something worth renting.

4 - An excellent game that is of above average quality. Great graphics, solid gameplay, and a good story, with maybe only a few technical problems holding it back from true greatness. Usually worth owning, but should maybe be rented first by those interested.

5 - An amazing game of superb quality. Everything from the sound, to the voice work, to the story, to the gameplay is incredible. Even a few minor bugs will go unnoticed, or will be forgiven, for the sheer amount of enjoyment to be had. A must-have for any interested.

That kind of thing. Obviously one could be more specific but that's just a quick run down of how I feel the -/5 system works and would/will personally use it.

Mr F.:
Right, I keep seeing this and now I must point it out.

What do you mean by "Should be average". What on earth does ANYONE mean by that.

If the average game gets a 7 then 7 = Average.

The problem is that the average game gets a 7.

So unless a site specifically states that they use a different system from other sites etc et etc, 7 = Average.

There is no "Should" with averages.

Most people who hate review scores seem to forget that. Yes, it fucks up the system, but we all have internalized it. If something gets a 6 we dont think "This is an above average game" we think, quite rightly, "This game is below average". Because it is below the average that has already been set.

You see where I am going with this?

I have nothing against review scores. I believe them helpful. Nor do I have anything against either metacritic, well the review scores from professional critics at least.

I guess I hold a hated view these days. Yes, I accept that the system is a bit broken. But that aint the systems fault.

Yes. Yes there is a "should."

What is the average of 0-10? 5. Start with the hard numbers, not the opinions.

The average game should, therefore, mathematically be *set* at 5. Just do it. Statistically, most of *anything* that has a large enough population, will be average. Reviewers should give a game that you consider to be completely average and unremarkable but still playable, a 5.

At this point, fairly good games will be then compared to this 5, and they will get around 7 or so...but right now, *average* games are at 7. So you have a massive, enormous range of "how much games can suck" (because 0-6 is apparently considered bad), and a tiny range of "how much games can be good," which are three numbers. In three numbers, can you really describe how much one game is better than another? No. Not really.

We messed this up because we started out by putting early reviewed games around the wrong number. Average games were given a seven. I like the theory that it's because Americans think on the grade-school report card system, but who knows why. So if your first unremarkable games are given a 7, then the rest of your games are going to be compared to that, and we're off to the races.

If you start out asking what your average game *should* get, mathematically, you will start at the right point.

Set your ratings at average 5.

"Review-Scores are HARMLESS" Pictured: Kirby aka. THE DEVOURER ...did anyone else find that ironic O.o?

The problem I have is that review scores are implicitly quantitative, not qualitative.

ANY review must be qualitative to have any meaning. You can't quantify an opinion, no matter how hard you try.
(and before some smartass comments about polls, that's counting a number of opinions within a population. The number itself cannot form the logical basis of those opinions. It's like saying "I believe him, because he used the number 3 in his review.")

So, I read a score of 10/10. This tells me you liked the game. Great. Fantastic.
Anyone can do that, even if it isn't true.

This tells me nothing about the game. Some people THINK it tells them is that it's an excellent game.
And those people are fucking sheep who don't actually think for themselves. Reviews are opinions. Not facts.

If I'm reading reviews to research a game, what good is that 10/10 to me? The READER?
As a summary opinion, it's valid (since opinions are subjective), but still quite useless. It's filler.
How do I know you aren't just typing that to suck off the publisher? How do I know you even played the game?
I can jump onto Metacritic right this fucking second and give any game a 100/100, without having played a minute of it.

OK, so lets assume you actually wrote a review, and you're attaching a score to it as a sort of summary.
Knowing that I JUST FINISHED READING/WATCHING YOUR REVIEW AND GET TO THE SCORE, I should already know your feelings on the subject matter, no?

A little number at the end isn't going to change what you wrote, and if you didn't write anything but that number, I logically have no reason to trust you. So, logically, I don't get why you would include a score except for either purely arbitrary reasons, or pressures from within your field.

In the absence of stupid business practices (like publishers using Metacritic as a basis for payment plans) I do agree that review scores are harmless...but because I think they're useless more than anything. Ignorance and the inability to logically recognize that fact is more harmful IMO.

I won't demand any critic change their style unless they are objectively misrepresenting their subject matter (flat out lying to the audience); as for review scores, I ignore them. They're useless to me because the score is arbitrary, and the scaling is arbitrary.

Jim, I think it's worth mentioning, I nearly pissed myself laughing at the end of the video. Great job!

I can both support the arguement you make here Jim, and see the opposite.

On the plus side, an x/X or X% rating gives you a general snap judgement of the overall feeling the reviewer had (or by their wording should've had) for the product. And I personally tend to read/watch/listen to the full review to put the score in context. For example, Angry Joe gave Asura's Wrath a 6/10 but with the BASoA and his review explained the hows and whys, which basicllay translated into (as i interpreted it) 'good game, not worth the full price' (in fact I'm pretty sure he said rent it/wait for it to price drop). I did the latter, enjoyed my 20 experience, even went on and bought the DLC.

On a related note, prolly wouldn't've shelled out for Sleeping Dogs had the 2BFP of it not made it look like so much fun. Could go on, but, yeah.

On the other side you have the AAA bought & paid for reviews where a 9/10 is the minimum (and is in fact now the score for an 'average game with a good ad campaign' &/or latest popular franchise sequel) and it seems to exist to be plastered in ads to sell more copies of Clone Shooter 2013 Edition. THIS is the kind of score we all want to be rid of... and since it's, for better or worse, the dominant for of scores, thats how we end up with this week's episode.

... Also I like the new murder gloves. Although Defoe needs some more input

It's the review score inflation that people are having issues with.

I don't understand why you would make a video to debate a point so very, very few people would defend.

For what it's worth, Jim, you're one of the few people who uses the review scores PROPERLY.

As in, beside the score, you say exactly what the score means. 8 being a great game with a few flaws, 6 being for genre fans only, etc.

A lot of other sites seem to arbitrarily slap the numbers on and make it seem that a 7 is a bad score, when it's really not all that bad.

Aurora Firestorm:

Yes. Yes there is a "should."

What is the average of 0-10? 5. Start with the hard numbers, not the opinions.

The average game should, therefore, mathematically be *set* at 5. Just do it. Statistically, most of *anything* that has a large enough population, will be average. Reviewers should give a game that you consider to be completely average and unremarkable but still playable, a 5.

At this point, fairly good games will be then compared to this 5, and they will get around 7 or so...but right now, *average* games are at 7. So you have a massive, enormous range of "how much games can suck" (because 0-6 is apparently considered bad), and a tiny range of "how much games can be good," which are three numbers. In three numbers, can you really describe how much one game is better than another? No. Not really.

We messed this up because we started out by putting early reviewed games around the wrong number. Average games were given a seven. I like the theory that it's because Americans think on the grade-school report card system, but who knows why. So if your first unremarkable games are given a 7, then the rest of your games are going to be compared to that, and we're off to the races.

If you start out asking what your average game *should* get, mathematically, you will start at the right point.

Set your ratings at average 5.

No that is a misconception, an error.
In a class of students, you can have the average be a B+ if your students are great. Or well any score that THEY AVERAGE, this doesn't mean the AVERAGE SCORE should be the mathematic average of the scale. SO if there is a clear evaluation criteria, and games accomplish to effectively fulfill most of the criteria, they should get the numeric score that applies, and not the average, even if in general most games succeed to a certain measure.

IE: if most games are GREAT, you shouldn't give them an AVERAGE score, because they are the average quality of games, obviously.

Jim my boy. It must be a slow season. I love your show but I find the irrelevance of this episode almost as irrelevant as the 10 billion review scores that are shouted at me from TV commercials.

But since were on the stupid subject. The only review scores I've ever respected come from X-play which I've watched since it was extended play and it was just Adam (huh? anyone? yeah!) This opinion has been formed over almost a decade of watching this show, understanding the nature of the game reviewers, and overall being entertained by the whole process. I trust them because at this point I know what they like. When some commercial flashes up with whatever the latest pop/rap crap stating "review junkie gives this a 5/5" or "game monster weekly 5 stars out of 5." Well you get the idea. That crap doesn't matter. You have to know and trust a reviewer to respect their ratings. Therefore most ratings don't matter. And vice versa from Jim's standpoint I don't know why he cares about the trolls. Because if they don't trust his review numbers then they don't really have much business reading his reviews.

On that note I've watched this show long enough to say Jim probably gives pretty fair reviews.

Keep on keepin on Jim. But please find some new topics, these past few weeks are topics that only the darkest corners of the internet are even privy to.

Aurora Firestorm:
I see much more complaining on the internet about how scores average around 7-8, and any game that got a 5 (which *should* be average) is seen as total crap, than how scores suck altogether. They're just weighted funny, giving them much lower resolution. Has any game ever gotten a 1? Ever?

Indeed. They do exist.

~deleted~

 Pages PREV 1 2 3 4 5

Reply to Thread

Log in or Register to Comment
Have an account? Login below:
With Facebook:Login With Facebook
or
Username:  
Password:  
  
Not registered? To sign up for an account with The Escapist:
Register With Facebook
Register With Facebook
or
Registered for a free account here