Jimquisition: Review Scores Are Not Evil

 Pages PREV 1 2 3 4 5 NEXT
 

Jimothy Sterling:
take their ball and go home as Konami did

...Wait, maybe I am out-of-the-loop here, but were you blacklisted by Konami because you said something mean about their game(s)!? Because if that's true I am going to be a little shocked and disappointed with them.

veloper:

This is a very good point. Prices drop all the time, especially on PC and length is only value for money (if the game is any good at all).
Game length should be mentioned somewhere, but it doesn't make an experience more fun. Some of us don't even want to waste much time.

Precisely. As a consumer product its length is relevant because you want the most for your money, so it should be mentioned. But as entertainment its length should have nothing to do with its perceived quality by itself.

veloper:

Trickier. A reviewer may want to reward originality somehow, but at the same time to a newcomer who isn't jaded yet, copy-pasta sequel X may still be worth their time.
Worse, a fresh experience may genuinely make the critic like the experience more. If this rule is to be taken to it's logical destination, then a critic would have to go something like: 'wow I loved this game! Hmm, maybe it's because it's something not done before, so I'd better substract a full point just to be fair to all the copycats that will soon follow'.

A reviewer should always score the game on their level of engagement; if the newness of the game is something that makes the game more enjoyable, that should definitely improve the score. But a lack of newness should not detract from the score. If NSMB2, for instance, was as good as the first game in every way it should not score lower simply because it is unoriginal. However, because you can get roughly the same experience with an older game, its value should be lower.

I think the problem is that most of the 'X out of 10' outlets are secretly working on an X out of 15 system, where 15 would actually be a perfect game and 9 or 10 is just a bit better than average. And people get so confused by this that now they get pissed off at people like Jim for using the number system for its purportedly intended purpose: as a quick way to tell people how good he thought the game is in comparison to other games in the genre (i.e. games he has scored either higher or lower or the same number to).

Am I the only one who got a sympathy chub with Jim's narcissism boner at his own gloves? Fantastic Dr. Strangelove-type times.

Siege_TF:

muffinatorXII:
i don't really have a problem with scores it's just that they make no sense. first of all i don't believe a complex opinion can be quantified numerically and if it could you would have to decide on a universal scale to use it on, which also makes no sense because different people value things differently.

and there is this weird thing right now where 7/10 is average

5/10 is funtional mechanically, which is not average, despite being the number between zero and ten, because consumers hold the industry to a certain standard. This is mostly thanks to the internet giving consumers the tools to have the developers by their dangly gubbins, and a game that does nothing more than function mechanically (like that X-Men game) won't turn a profit (it didn't). So 7/10 is 'average' in that it meets our standards, as in, it's entertainment that's entertaining, meaning it does more than function, which is what we expect, and have every right to at sixty dollars a pop.

5/10 is a car that runs, 7/10 is a car that runs well, and has air conditioning and a radio. It may not park itself, it may not be a hybrid, it may not have heated seats and a damn GPS, but it's what people consider 'average' in spite of cars not needing A/C or a radio to function.

It's not that complicated.

I can definatly see your point but these extras have become the standard due to the tech and goals within the industry. Im not saying that every game MUST include them, its very much up to what the creative people behind the game want to acheive, but as standards increase the bar gets pushed up. Isn't this how its always been? Im aware im being quite general here but this is artistic oppinion were talking about, even if I have no problem with the use of numbers everyone has there own way of doing so as this isn't technically maths as there is no universal formula behind it.

E.g. If I went out to buy a car and it didn't have a radio id mark it down, granted only slightly. I'd say at least 80% of cars these days have a radio, its become the standard, the absense of one would be a negative not a plus if it did have one.

Scores are great, how else will the robots know what is good or not when they kill all humans and take over the world?

Eh...Jim's points about why we should keep scores are all valid, but the very concept of scores is something that has never sat well with me.

I mean, think about it: you're trying to sum up an emotional concept (how much you liked a game/how it made you feel) with a logical one (a number). This is just something that strikes me as bizzare, like describing the colour of the room in terms of how sonorous it is. I mean, it seems to me to assume that you can give every soaring, wonderous moment, every grim, disastorous failure and every annoying glitch an exact numerical value. "Yeah, it was deep and incredibly moving, but I only cried for 20 minutes, so that's an 8/10." The incredibly many ways to play each section of a video game drastically compound this issue. It doesn't really seem that I could even rate a small section of a linear game more accurately by giving it a number than by describing it how it is in more general terms, since the number would just be summing up all the information in the level about how I felt about tackling it, rather than giving a fuller picture and letting readers/viewers come to their own conclusions, with all the information at hand. Furthermore, even if you could objectively measure the good and sum it up numerically, it still wouldn't be that accurate because it assumes that all good is the same kind of good. I enjoy tragedy for very different reasons to drama, expect very different delights from a stealth game than a shooter and wouldn't try and hold my standard of what is good to someone else's wildly different idea of what consists of fun. In the end, this whole concept boils down to trying to place an objective measure on a subjective bit of information, and that's why it's iffy for me. Yes, I know you'd have to be stupid to take a number at the end of a review as fact (after all, it is still opinion, even in number form), but it still feels like the wrong way of summing up to me.

Mortamus:
I, for one, welcome our new Sterling overlord.

In reference to the removal of scores, it's as old an argument as any. It just has a different title. Generally, the idea of "Take away their [tool] and then they can't do [deed that is deemed awful by it's author]" has little to no sense. If someone wants to do it, they'll find another means to do so. Plain as that.

Sterling overlord you say? So does that mean instead of ruling over us with a iron fist he'll have a SILVER one? ;)

Also, I agree with what you said about the issue.

I never understood how people could cry foul at an eighty percent or a solid B. As much as I hate throwing the "Fanboy" label around, there's some people who can't understand that everything - absolutely everything - should stand up to criticism. If you can take a game you honestly adore and still find niggles and small issues with it, you're doing your job adequately. If you're so blinded as to declare this or that game the epitome of perfection or are unable to understand that review scales change from site to site and reviewer to reviewer, then you don't understand the point of criticism in any shape or form.

No game will ever be perfect, and a eight or nine out of ten still denotes a very respectable and even commendable success. I'll buy games that sink as low as six or seven, sometimes, and I understand that all reviews are ultimately subjective.

Consider this: a friend of mine absolutely fucking loves Dead Island. He compares it to a Mle-centric version of Borderlands, and he loves the way combat can be fairly difficult or unforgiving. I played the same game for three hours and couldn't find a speck of enjoyment in it. It felt badly put together, poorly structured and thought out.

Who's right, then? We both are. There's no point in him trying to convert me, just as there's no point in me calling him a poopyhead for daring to like what I don't like. You can disagree with critics, and critics *will* disagree with you.

Yes, there's always going to be thorny issues like paid-for reviews and the general impression that payola is a common practice in the industry - allegedly, at any rate - but I'd still rather think that professional reviewers are hired for their ability to be honest about their preferences. This is why most pro outlets aren't just the voice of one guy testing out various games - a JRPG buff might have zero patience for a Western shooter or a sports sim. Making sure you nab reviewers from all corners of the gaming rainbow keeps things fair and balanced.

Okay, I'll give you that it's not the fault of the scores that people so blatantly misuse them, and yes I would love some kind of actual stand against these people beyond "well this guy is a dumbass, time to put them on my block list so I don't have to read more of their stupidity". Personally though, I still find review scores to be useless. Saying that a game is a 7/10 tells me one thing and one thing only: that whoever gave it a 7/10 liked the game despite what they felt were only minor faults. It does absolutely fuck-all to tell me if I'd actually enjoy it. I actually have to read the review itself to find that out, and by the time I've finished the review, I don't need a score anymore. It's just an extra summary put in after the written summary that uses a number instead of words.

I still want to repeatedly kick the genitals of every person who insists that because a game got good scores I'm not allowed to dislike it and it's better than a game I did like if it got higher scores. Just today some complete and utter dipshit on YouTube, in response to me saying I didn't like Sly Cooper while also recommending Ratchet and Clank to a friend, insisted that I must have never played Sly Cooper because it got better review scores than Ratchet and Clank and thus is a better game than Ratchet and Clank. FUCK. OFF. That is not what review scores mean you useless cretin!

Blood Brain Barrier:
Jim should do audio clips rather than videos. I don't need to watch streams of clips from Japanese games I don't care about while at the same time listening to completely unrelated streams of Jim's occasionally funny/entertaining monologues.

...Then don't. This may come as a total surprise to you, but you don't have to watch the video just because it's there. You can open a new tab and just listen to the words while you look at something else. You can minimize the browser, turn off your monitor, or get up and do other things in the room while the audio plays.

Meanwhile, those of us who like the video, especially when it's a situation where the game clips help drive the point home (such as "Monster Boobs And Plastic Children", where I never would have known that the volleyball game he was talking about was that creepy if he hadn't been showing clips throughout the episode), can still watch them instead of having them taken away because the almighty Blood Brain Barrier dislikes them.

Lvl 64 Klutz:
It depends on how you look at the 10 point grading scale. If you look at it like an academic grade, then 7/10 *should* be the average/mediocre game. That's how most reviewers see it, even. Just look here on the Escapist where 2 and a half stars is usually accompanied by a "don't bother" recommendation and anything below that is pretty much considered garbage.

Which is wrong, because if 7/10 is the average/mediocre game, then you have only 3 higher numbers to give a game a better score and 6 smaller numbers to give the game a worse score. This "7/10 means mediocre" nonsense is why people cry foul at 8/10's. 8/10 does not mean slightly better than mediocre, it means very good or great. 5/10 is mediocre, not 7/10.

PunkRex:
E.g. If I went out to buy a car and it didn't have a radio id mark it down, granted only slightly. I'd say at least 80% of cars these days have a radio, its become the standard, the absense of one would be a negative not a plus if it did have one.

Exactly. For claiming that it's "not that complicated", he sure got it dead wrong and you got it absolutely right. Having a standard feature does not mean you did better than average. To go back to the car comparison, like you said, a radio is expected in every car, and having that is not worthy of a higher score. Not having one is worthy of a lower score, though. You'd have to go above and beyond with the radio to get a higher score, it would have to be more than a standard FM radio with a tape deck and/or CD player. Maybe it has a 5 disc CD changer, or maybe it has built-in satellite radio so you can listen to radio stations that aren't 75% idiot DJs talking about shit you don't care about and commercials. Then you get a mark up. Having a basic, average radio gets you a basic, average score. And not having one at all marks you down.

These "7/10 is the average because it should be the average" people really need to try out common sense sometime.

Review scores in and of themselves are not bad. They are actually really great. They give you a quick glimpse of whether a game is good or not as viewed by a certain person or site. I like the */5 scores that the Escapist gives their games but sometimes I already know a lot about a game, I've been following it for months, so all I really need to know is whether it was good as everyone thought it was going to be. If I see it gets a really bad score then I'll usually look deeper into it and find out why and see if the problems listed would hinder my experience to the point that I wouldn't want it.

The real problem with review scores is how much emphasis is put into them. People will point to a game and tell you how bad it is with no other explanation then, "It got low scores." Companies focus on trying to get a high Metacritic score and will make stupid decisions because of this. People will Review Bomb a product because of some perceived slight. And quite possibly people will give something a good score just because they are paid to. Scores become this ominous thing that holds such a tight grip over the video game industry, and so many call for its destruction.

In reality the thing that NEEDS to happen is we need to change how we score games. There are many places experimenting with this in various ways. You can separate the game into categories such as Gameplay, Aesthetics, etc. and give separate scores for each of those. You can change the criteria on how a game is judged. You can give pass/fail grades to the games. There are a myriad of different techniques being tried, and I hope we find one that works the best, simply because the "Overall" grade that we tend to give games now a days doesn't really do them justice. But then again that's why you should read the review as well.

personally i always liked X-play's old standards.

5/5 exceptional, everyone should play this game.

4/5 good, worth your money

3/5 average, worth renting, or buying if your a fan of games like this

2/5 bad, if you spent more then ten bucks on this, you were ripped off.

1/5 horrible, don't touch this sh*t with a twelve foot pole being held by someone else.

its nice, simple and keeps people from getting upset over that 8/10 crap.

xPixelatedx:

Jimothy Sterling:
take their ball and go home as Konami did

...Wait, maybe I am out-of-the-loop here, but were you blacklisted by Konami because you said something mean about their game(s)!? Because if that's true I am going to be a little shocked and disappointed with them.

Yes, they got upset with me because of my Konami Jimquisition video, as well as some negative reviews. I am now less than dirt in their Eastern office.

Yeah, numbers totally work.

The 5 out 10 = bad, 7 out of 10 average and 10 out 10 good game "and the publisher also made a lot of advertisement on our site" system is flawless. The last spunkgargleweewee incarnation CoD Black Ops showed that.

Review scores are a stupid idea, because corporate chimpanzees now use the for team reviewing and bonuses. EA does it for example. So all hail to metacritic. Of course now everybody wants good review scores, so publishers and developers do everything for a good review and the system is getting more and more corrupt. Because videogames are serious business.

And because most game reviewers are not earning as much as the TV or movie critic, bribes have a bigger impact. Of course there is no money getting paid, just add space, early access or free trips to game presentations.

Jeff Gerstmann once gave a bad score to a game, gamespot and Eidos already had made a deal about and we all know what happened. Review scores made the system corrupt and that's why most people in the business want to keep them. Because as long as the do, publishers and developers will do everything to make reviewers feel like a very special snowflake and will pay good money to their employer. So review scores will ensure that the spice is flowing.

And the spice must flow.

Wait, Jim takes our advice on fashion tips? Then I say he should do an episode nude! Muhaha! My evil plan is coming together! Now I just need to get the duck and a ray-gun.

Anyways, back to more serious business. I actually really like review scores. Why? Because they normally are a general indication if a game is really good or not. I'll normally look at the review of a game and read through it if it's something I'm already considering picking it up. But if it's something I don't care much about, I normally glance at the summary, the score, and move on. But if it's almost universally praised as a good game I might decide to check more into it. This has happened with quite a few games in the past and it's almost never lead to disappointment. Now, I might be missing out on a lot of games I might have liked because I didn't read the full review. But given my current financial situation (Read: Because I'm too piss-poor to be able to afford many games) I have to be more selective on what I pick up.

mjc0961:

Lvl 64 Klutz:
It depends on how you look at the 10 point grading scale. If you look at it like an academic grade, then 7/10 *should* be the average/mediocre game. That's how most reviewers see it, even. Just look here on the Escapist where 2 and a half stars is usually accompanied by a "don't bother" recommendation and anything below that is pretty much considered garbage.

Which is wrong, because if 7/10 is the average/mediocre game, then you have only 3 higher numbers to give a game a better score and 6 smaller numbers to give the game a worse score. This "7/10 means mediocre" nonsense is why people cry foul at 8/10's. 8/10 does not mean slightly better than mediocre, it means very good or great. 5/10 is mediocre, not 7/10.

Perhaps I shouldn't have included the term mediocre, because you're absolutely right. Halfway between unplayable and perfect would be mediocre. But if we're looking at what should be considered "average" both mathematically and a standard of quality, then our medium should be aiming higher than halfway between unplayable and perfect.

That was the point I was trying to make. A game with 6/10 shouldn't be considered better than average. It should be a game that has it's fun moments, but definitely has room for improvement.

DVS BSTrD:
I don't care about scores, I care about the reasons behind them.

And that's the /thread for me.

If people are that gullible to be misled by scores about games that they haven't played, well, I can't help anyone. There's a reason that I don't trust scores anymore. Hint-it's not the inflated average.

My compromise has always been to use X/5 instead of X/10 or X/100. When it's X/5 it's easier to interpret 3/5 as average than 5/10 or 50/100. It makes the review feel more like a suggestion than a grade on a test.

I personally dislike review scores because I think they devalue the review its;ef; and let us not forget that Geff who worked at Gamespot who was fired for giving Kane and Lynch a low score.

I agree review scores aren't bad in itself it's just that ppl put way too much importance in them. (metacritic, the industry, gamer)

Imo review scores are a symptom of a consumer market, where there are so many games all fighting for our money that most gamers put every little effort into choosing what games they buy and play. I a perfect world ppl would do their homework into discovering new games. Instead most ppl don't bother, they'll just play sequels and the same genres over and over. Where only the most hyped games are successful only ppl repeatedly tell them it's a good game, same goes for games like FTL ppl are interested due to the buzz surrounding it. (even if it's a good game) Thats most gamers who read reviews already plan on buying the game anyway.

This is why games that are clones or have gimmicky game mechanics sell best because that's what ppl understand, while so many good and unique games that might be critically acclaimed like Vanquish, Okami, Beyond Good and Evil Skull Girls and Akia Katana go under the radar and bomb at the retail counter. Then years later ppl are falling over themselves to sing it's praises and demand sequels even though 99% of them either ignored it of slagged it off when it first went on sale.

The only arguments I've seen against Final Fantasy XIII are ones that seem to come from people who have never actually played the game. Even /v/ has favorable things to say about it. I'm not saying it's great, but it's far better than, let's say, Final Fantasy VIII. Or earlier titles that had endurance tests that tended to rely far more on luck than anything substantial. I have never been so bored or frustrated playing a game as I have playing Final Fantasy VIII.

No wait, that honor is reserved for Mafia. Oh how I love waiting in traffic while the AI sorts out its pathfinding under a street light. FUN FUN FUN.

I do agree that Metacritic is more or less harmless by itself, but you have to remember that there are retard publishers are there who think it means something. In a way, it does. Generally, green scores == money.

Oh great, Jim's become a Saturday morning cartoon villain.

Bocaj2000:
My compromise has always been to use X/5 instead of X/10 or X/100. When it's X/5 it's easier to interpret 3/5 as average than 5/10 or 50/100. It makes the review feel more like a suggestion than a grade on a test.

It's weird how a 3/5 still sounds like a good game but a 60/100 is usually passed over. I guess with little numbers it's easier to focus on the fact that it still has 3 points instead of it not having those last 40.

I didn't realize that this was an actual problem. I guess there are whiney idiots protesting everything these days.

Jimothy Sterling:

xPixelatedx:

Jimothy Sterling:
take their ball and go home as Konami did

...Wait, maybe I am out-of-the-loop here, but were you blacklisted by Konami because you said something mean about their game(s)!? Because if that's true I am going to be a little shocked and disappointed with them.

Yes, they got upset with me because of my Konami Jimquisition video, as well as some negative reviews. I am now less than dirt in their Eastern office.

I remember watching the news once and the interviewer asked the person, "How does it feel to be rich" to which the person replied, "It feels great". It was bullshit because of how that person got to be soo rich. He ran a Ponzi scheme. But of course the journalist never would have gotten that interview in the first place had the guy thought he was going to get negative PR from the experience and had that interview turned serious the guy would have just walked out.

Journalists have the responsibility of calling saints, saints and assholes, assholes. Journalists are not supposed to be used as PR tools. Journalists are not pals of companies or certain individuals. If you call out Konami for doing what Konami does and they choose to back that up and, as you put it, take their ball and go home then they only have themselves to blame when people get upset at them.

We don't, or shouldn't, want Konami, EA, and others to fail, but when they do fail and fail hard it's because we wanted them to succeed that makes us upset and so vocal. It is in the best interest for all when they do good. If they treat you less than dirt because you did your job AND theirs then the problem isn't you. The defiance in not realizing that they are the root of their own problems is like Konami shit the bed and when people found out they just put on a diaper then acted to treat others with contempt out of embarrassment. That diaper is not a solution to the root problem that they are still shitting themselves.

At the end of it all they just create this cycle of shit and disdain followed by more shit and disdain and that is truely sad.

Another:
I don't mind review scores, if it's a true scale score system.

In a scale of 1-10, five is average. Not 7 or 8. 7's and 8's can still be really good. Iv'e enjoyed a fair few games that have received such.

I'm rather irritated with that point. 5 does not have to be average. 5 just happens to be the middle number. I don't know about how you get graded but for me, if I scored a 50 on a test, I'm not going to complain that I failed because 50 is, "Average". To say that a 7 or 8 is average is EXACTLY as arbitrary as saying that 5 is. No, 5 is not inherently average. What is average is dependent on the kind of scores most games get. If average games get 7.5, then 7.5 IS average. Like...you know...by math.

All you have to do is think of review scores like letter grades, and everything makes sense. Whining over AAA games are just like the class genius who is obsessive about not losing a single point. To not accept review scores as following a trend like this is just pointless stubbornness that the rest of the world doesn't want to conform to your unit of measurement. You might as well tell someone that they are stupid because that road trip was NOT 15 Kilometers, it was 9.3 miles damnit, and your dumb for saying otherwise! If you must insist on 5 being average, just do the conversion in your head.

Scores are fine. I like scores. Even on the sites where 7/10 is "average," if its kept specific to the site its on then I don't really mind at all. I love Destructoid's scores that are very clearly marked and have purpose, I love Giant Bomb's scores which have one star through five star mean a very specific thing. Scores are great.

Metacritic, on the other hand...

Honestly. I'd be fine with review scores that don't align with each other (5/10 on Destructoid isn't 5/10 on IGN) if Metacritic weren't so incessantly popular. If Metacritic didn't weigh certain outlets more heavily than others based on ad revenue. If Metacritic wasn't the biggest point of call that publishers use to gauge critical opinion, to the point where jobs are on the line due to Metacritic scores. If Metacritic didn't convert - or in some cases, outright guess! - what the scores are out of a percentage. Giant Bomb themselves have said, hey, our five star rating doesn't convert to a out of ten score, and an out of ten score doesn't convert to a percentage! Since they only give whole stars. So when Metacritic says "Giant Bomb gave this game 80%!!" that's disingenuous. And then you get, say, Fallout: New Vegas, where the devs lost their bonuses because it rated below... what, 85? It's bullshit.

So scores are good. Compiling all those separate systems that every website uses almost on a writer-by-writer basis, and attempting to compile them and aggregate them, and then weighing the importance of jobs and bonuses and contracts on top of it? That's bullshit. If ever an argument for abolishing scores was strong, it would be that it would also mean the death of Metacritic. And that would be a victory.

Xanadu84:
I'm rather irritated with that point. 5 does not have to be average. 5 just happens to be the middle number. I don't know about how you get graded but for me, if I scored a 50 on a test, I'm not going to complain that I failed because 50 is, "Average".

I believe he's referring to 5/10 meaning "of average quality." Not the maths of it. On an opinion scale of 1 to 10, 5 should be the middle opinion, and if you weigh 1 as "worst ever" and 10 as "perfect," then 5 comes out as "average." Of course once you apply maths to the score system then things start to get a bit skew-whiff, but I think most people accept that the qualities associated with number scores - especially stars - are permanent, not dependent on a bell curve or what have you.

The problem people typically have with game scores have nothing to do with game scores. The issue is that people seem to look at the game score like it is some kind of enjoyment meter, where the closer to the high end it is the more the person is going to enjoy it. This really isn't the case, as there are interesting games that score only a 7/10 just as there are cookie cutter games that get a 8/10 to 10/10. One has to account for their own personal tastes.

Jimmy my boy, I dare say you were looking at that glove like you've got a hot date tonight...by yourself. :3

It seemed like Jim was repeating himself towards the latter half. Got kinda boring.

Heil Jim!
long live his glorious empire of video game commentary!

Great vid as always, Jim.

I read reviews for their entertainment value, and to try and get into the mind of the reviewer, carefully reading between the lines to parse their likes and dislikes and gain insight into their lives, making them seem more human and not just a sack of meat connected to a pair of hands at a keyboard. I buy games not because of review scores, ut based on whether or not the game itself looks good to me. Also, how much I've got on my plate at the time and whether it's good enough to justify adding to the already-ungainly-at-the-very-least pile.

One thing I do NOT do is meticulously scan the text of each review for spelling and/or grammar errors that I can use as justification for mental gloating. ...No, I definitely don't do that. _

Also-also, NEW PAIRING!! Jim x Black Gloves. It's canon and don't you deny it!

I do the same thing as Jim here. I look at the score then read the article. But I would never look at a review score and then buy a game. And thank god, I would have bought some real lemons, and conversely missed some great games.

I think people don't like scores for the same reason Yahtzee doesn't like using scores: he doesn't think a complex opinion can be properly expressed in numerical terms. I don't have that much of a problem with review scores, but I think too many people take them WAY too seriously, and THAT'S why people want to see them gone.

Yeah, just because one or two prominant names like Yahtzee have decided not to score things on a numerical basis, people have latched onto the idea that scores just don't have a place in the world, despite movie critics and the like using them effectively. Score inflation and the industry's use of metacritic aren't sufficient reasons to abolish scoring entirely, just wake-up calls to use the system more responsibly.

 Pages PREV 1 2 3 4 5 NEXT

Reply to Thread

Log in or Register to Comment
Have an account? Login below:
With Facebook:Login With Facebook
or
Username:  
Password:  
  
Not registered? To sign up for an account with The Escapist:
Register With Facebook
Register With Facebook
or
Registered for a free account here