Why We're Using Review Scores

 Pages PREV 1 2 3 4 NEXT
 

I've never paid much attention to the reviews the escapist offers, as I don't have a lot of faith in reviews altogether. Games, movies, books etc. will always appeal to some people more than others and this shows in most reviews. For example, I went to see Yes Man with a couple of friends and I had a great time. The critics apparently didn't, calling it unoriginal and stale.

I have seen some video reviews on here though, and they do give a little insight into what a game feels like. Maybe I'll try reading one next time.

If people only want a score, then why bother with writing anything in the first place? You might as well write the review out, print it, fold it up and wear it as a fetching pirates hat so you can still feel like you actually did something worth while. I hate scores, they don't tell you anything about a game. they're totally pointless.... If you pardon the pun... Or don't.

I oppose.

But I suppose I do so chronically in respect to most things.

So this is to appease metacritic?
Or do I have to use a different Stega-sheet.

It's very hard to be in favour of scores, and this seems like a slippery slope. 5 stars is better than a percentage though, those are just silly. As long as the quality of writing doesn't decline, I'll let this slide.

I understand the need to sell out. You're trying to turn a profit after all. Still.. it always does pain me to see a well-specialized niche player dumbing itself down to take on the mass-market.

Yeah, you may fully intend your reviews to stay the same, but the next step is when you start doing your site metrics and seeing, "Man, it seems most people are only looking at the last page of the review.. if we start doing shorter reviews, we can pay writers a little less for each and get more reviews in at the same time," and you'll rationalize that as benefiting even more of your audience because you'll be able to provide coverage of an even broader spectrum of the gaming/entertainment market and "Hey, if one of those happens to check out some of our indepth stuff, then we've done a good thing and our consciences are salved"

Except in the meantime, your in-depth stuff is becoming increasingly buried under the weight of web-comics, anime, machinima, tv-reviews, celebrity designer interviews (which then lead to celebrity player interviews, and then just plain celebrity interviews) etc., and while People is a good magazine for what it does.. it's already there, and we don't need another.

A foolish consistency may be the hobgoblin of small minds, but I've yet to see any reasoning suggesting that consistency on this issue was foolish in any manner.

John Funk:
As Russ said, the only thing that's going to change is that there will be a little gold star at the end of the review. We like our format. We think communicating what it's like to actually play the game is the most important thing, not whether a game gets an 8.6 or an 8.5 and how you can possibly quantify that.

So... why bother changing? You (that is, the Escapist staff rather than you specifically) think that's the most important thing and we agree, obviously, that's why we're here. Frankly, the addition of review scores seems like nothing more than a desperate cry to be included by Metacritic, as backed up by the last paragraph of the first page... and that's something I would never have expected. Everything about The Escapist is far superior to the rabble, so why join them? It's a slippery slope.

The rest of my thoughts are well put in the quotes below.

Skyweir:

Dorkmaster Flek:
Well everybody else pretty much said what I was going to regarding this. You don't seem to have anticipated the possibility that you will actually lose some of the people who appreciated the fact that you never pandered to the lazy asshats who can't be bothered to actually read the review and want a simple distillation of a complex opinion. None of your arguments convince me that this is a good thing, and frankly, I'd rather you indirectly tell anybody too goddamn lazy to actually read to just fuck off. It was one of the reasons I actually appreciated the content on this site over the big sites like IGN and Gamespot.

This is my feelings as well. Though I am by no means a frequent poster, I did use the Escapist for reviews.
If you intend to fall in line with the rest of the reviewers out there, there will be no point in reading yours over that of any other. Numerical grades tend to draw the eye and "force" people to skip to the end of the review. As a result, the writing deteriorates, as the writers realise that there is no need to exert themselves. Do you think places like Gamespot became what they are today over night?

There is no need for the Escapist join the dark side, we have enough of those sites out there already.

EDIT: And the reasoning that this might make people that normaly just look at scores read the review is pretty weird. I think you might have cause and effect backwards there. People that read only scores can hardly be sucked into reading the review, since by their definition they read the score and move on. The quality of the unread text is immaterial.

Scores are meaningless, so you can do whatever you want with them. Try to be different or jump on the bandwagon: it's all the same to me. It's not a part of reviewing that I care about.

My problem is that I fail to find this "experimental" quality that's been mentioned several times in the article. All the ones I've read have been by-the-books. I can't name a single memorable review here, despite the high quality of your other content. I can sympathize with your ideals, I just haven't seen them applied yet.

I try to keep an open mind, so I'll gladly take a look if someone points me towards something special I could've missed. I'm also thinking of examining one or more of your reviews here to illustrate my point, but at this time it seems like doing someone else's job instead of them.

Either way, it's much appreciated that you explain your decisions and share your beliefs with the community in such a way. You're good people.

Why join the rabble? Because we want to stay here, providing content for you guys. In order to do our jobs better, and provide better content, it's become clear that this is one battle we have to concede. If we want to keep fighting the bigger fights, sometimes we have to give a little ground. And this is one of those times.

Nothing else about the reviews is going to change. I certainly understand the concern, but please don't automatically assume that we're suddenly going to stop caring about how we do things simply because we've added a score.

...and it's always stars.

I'm not happy, mostly because I thought of the Escapist as the proof that a review didn't need scores to be meaningful. But I can't stay mad at Russ while he's wearing that adorable DA avatar.

You're off the hook, Russ... this time.

Susan Arendt:
Why join the rabble? Because we want to stay here, providing content for you guys. In order to do our jobs better, and provide better content, it's become clear that this is one battle we have to concede. If we want to keep fighting the bigger fights, sometimes we have to give a little ground. And this is one of those times.

Nothing else about the reviews is going to change. I certainly understand the concern, but please don't automatically assume that we're suddenly going to stop caring about how we do things simply because we've added a score.

Seriously, now, this is the worst PR for an editorial change I've ever seen.

I'm not saying you should post banners all over the site with "The Escapist: Now with more STARS!!!" on them, but at least pretending you all don't absolutely hate the notion of adding a rating to reviews would help.

I mean, look at this thread, the reaction isn't even entirely negative. Why do you guys sound so apologetic? I appreciate the concern for you readers' feelings, but it's your site and you can fill the whole thing with constellations if you want, or if you need to or if it's going to make you more profitable. It's perfectly fine. In fact, all evidence suggests that adding a rating will bring new people without driving current readers away. It really has no downside other than the very minor possibility that some of the new people coming won't bother to read the full review (which they probably weren't doing in the first place if all they wanted was a quick one-word summary).

Of course, like I said before, I was mildly annoyed by the lack of a numerical score anyway, sice I actually thing they bring something useful to the review, but I don't think most people who liked the number-less reviews would disagree with me, either.

It bothers me that an article called "Why We're Using Review Scores" gives not a single reason why you're using review scores. You give plenty of reasons why we shouldn't see this as a bad idea, but not a single one as to why we should see it as a good idea. And there's a pretty big gap between outright bad ideas and actual good ideas, a gap filled with ideas that are pointless and counter-productive.

What you've said, and even repeated a few times, over the course of this article is that changing with the times isn't bad, and that you're going to try to hold on to your integrity, and that you're willing to overlook all the downsides. But you haven't made it at all clear how the times have changed to warrant this (Was there not just as much of an argument for review scores four years ago?), or whether it might be easier to preserve your integrity without review scores, or what you're getting out of this that's worth overlooking the downsides for.

To be perfectly clear: I actually like review scores, and have no objection to them whatsoever. It's this article which I think is silly, and if anyone dislikes review scores I doubt this'll persuade them.

Well, to "long time listeners" this isn't a surprise. But you have to promise that when you start to make too much money, you'll return to your roots and go back to a weekly publication with that stylish magazine layout. That was the cat's ass and still would be. ;-)

Dark Templar:
You want to put in scores because some people only want the score? If hey won't read the review and just look at the score, why do you even want their traffic? I know if I wrote something I would want it read and if you only want some artificial number to decide things for you then I don't want to share my opinion with you anyway. Don't read, piss off back to sites like IGN.

I might just be a snob, but that how I feel. The kind of people who will come here just to see the score are not the ones we want here anyway.

I agree with all of this. I scoured this article looking for logical reasons to include review scores and could not come up with much beyond, the industry likes it and so do some gamers. If I understand this article correctly, it saddens me.

I also liked what you said about not wanting those people here anyways.

As a wise man once said "oh, dis is sad day!". One of the things that makes the Escapist unique and separates it from the crowd has been lost. I suppose adding review scores doesn't really hurt what already exists and may encourage new people to join but it also invites in the kind of elements that are often keep out of the Escapist in order to differentiate itself and accumulate an audience from a different crowd. Well, here's to hoping this doesn't backfire and destroy the site eventually!

I just hope the starsystem doesn't devalue over time as seen on some other sites.

Maybe you should use the starsystem on different criteria like graphics, action, story, voice/music and so on - thus having several stars or sub-stars before a final judgement.

Or maybe the fifth star shoud be graphically smaller than the first four. Because the fifth is the final bonus hard-to-get star.. not like the other four stars. But its still takes up 20% of the visual grade!
So I suggest that you make the first four stars the same size and the fifth a smaller one to be given on special occation.

While I too deplore the move to add a rating, at least you have followed the advice of Clint Hocking and gone with "Five Stars and the Truth". It will be difficult for you to game that system, and I applaud it.

"Be honest, and unmerciful." -- Lester Bangs, Almost Famous.

JeppeH:

Maybe you should use the starsystem on different criteria like graphics, action, story, voice/music and so on - thus having several stars or sub-stars before a final judgement.

Oh God, that's exactly what I think any website should avoid; breaking down an object into smaller pieces. To quote Salen and Zimmerman: "A system([game]) is more than the sum of it's part."

ShadowKirby:
Oh God, that's exactly what I think any website should avoid; breaking down an object into smaller pieces. To quote Salen and Zimmerman: "A system([game]) is more than the sum of it's part."

Quoted for truth.

Less detail is better here, guys. If you have a percentile system (1-100 or 1-10 with decimal points allowed) you can game it endlessly. "Oh, you want a higher score to imply you're better than this game? OK, we'll make you an 8.6. We're all happy!"

Five Stars and the Truth. If you must have a rating, that's how to do it. The Escapist chose the right system of ratings, if it must have one.

Strange that you don't seem to mention that the increased visibilty (at say, metacritic) and higher visitor count are the reason this decision has been taken. Also, given that you've got so many years of quality behind you you can rely on the core fanbase to stick around.

I thought it was for Metacritic...Honestly.

So, in short. "We do it because it is industry standard."
Why use 500 words when 10 will suffice?

One thing I would like is that if you must do it then at least give different segments different scores. So that the number browsers can get a idea of if the story is good, if the graphics are decent, how it controls. Not just "Overall: #X" Each segment.

Oh and please, realise something that most other sites seem not to which explains why I never care about review numbers ever. 5 = average. Not 7-8. Most sites treat it as 8 = average and anything below is not worth the time of day. If a game ever scores 5 it is considered a steaming pile of horse feces. Kind of fucks up the whole point of a general idea score.

For those asking for a more granular system with separate ratings for graphics, gameplay, etc. I urge you all to please read Clint Hocking's Five Stars and the Truth -- He goes into the whole mess of how videogame ratings are insane. While different ratings for each part aren't his focus, the same logic begins to apply: It's too much detail. It's also very much against the holistic focus of The Escapist, where a game is assumed to be more than the sum of its parts.

Keep it simple. If there must be ratings, make the ratings easy.

Clint Hocking's Five Stars and the Truth can be found at his website.

http://clicknothing.typepad.com/Design/hockingc_GDC09_Microtalk_FiveStars.zip

While I personally do not like it, I believe you have justified yourselves in this respect. It won't hinder the experience, it'll just mean that more will enjoy it for different reasons.

On the plus side, your scores can now be factored into Metacritic, can they not?

Yay for Metacritic links! (I was about to humourously post something more snarky about selling out to Metacritic, but that joke's been done to death.)

As long as the long-form experential reviews stay intact and don't get dumbed down to feed the Score Monster, then this'll work out fine. (Although it might necessitate more banhammering in the forums.)

squid5580:
So the reason you changed is why exactly? Because everyone else is doing it? Because you can? Because you just got sick of the debating it over and over? It is a shame to see something like this happen. It will be even a bigger shame to watch the posts in the reviews degrade to "well you gave game X 2 stars and gave game Y 3 stars when game X is a better game." Or the classic "This game deserved X stars"

And where Bobcat may not have concerns about the integrity I do. Your reviews used to read like a user review. A fellow gamer reviewing a game and giving thier thoughts and opinions of it. This put your reviews miles ahead of Gamespot or IGN's since it sounded like you were playing games for the love of the game. Not for the love of a paycheck. It looks like now you will have to play the games you review. Not want to play them. And that will effect the integrity of the reviews. Afterall it is hard to give a game a fair chance when you really want to be playing another.

im glad im not the only one that noticed that no reason to switch to giving them a hard rating was given

they reevaluated the situation and changed their stance? why?
i dont necessarily mind a score, but i dont pay attention to them at all and can see the potential abuse of integrity.

one of the main reasons i read the reviews and articles here is because they are so good and previously did not rely on scores to do the talking

edit: the idea that a complex and multifaceted opinion on an equally complex game would be shoehorned into a simple number bothers me, what if you start writing reviews based around your scoring method....instead of a complex multifaceted opinion we would get "this section was 3 star, that was a 2 star bit..."

also, scores force comparison where there should be NONE. game a gets 4 stars, but game b gets 2...there will be complaints and forced comparisons.

Boo...

Even though I follow review scores, I trust Kotaku and Escapist especially because they don't have a magic number at the end.

G4 gets an Ok for the similarity to the Buy/Rent/Avoid scale

Oh, and for those wondering why?

I'm not part of the editorial staff, so I can only speculate, but I will, as others have: Metacritic. It brings in hits.

Why now? Again, I can only speculate. But I'd bet this had something to do with it.

http://www.gamepro.com/article/news/213659/crispy-gamer-closes-editorial-department-ceo-resigns-in-protest/

Crispy Gamer was a site basically trying to be the next Escapist. It offered similar commentary and review style. And it just, effectively, got gutted.

Guys, you need the money. It's not an option to ignore it. A cautionary tale:

I used to write for White Wolf Games Studio. Pen & paper roleplaying game developer. Their game, Wraith: The Oblivion is critically regarded as possibly the best game they ever made. It was passionate, experimental, serious in its intent, dark and beautiful.

And it flopped, crazily. Why? Because it never found a market.

They then brought out another game, Hunter: The Reckoning. It was mocked for its used of gung-ho violent artwork, which didn't fit the tone of the actual game. For insisting on putting in new powers in every book, which was blatantly commercial. There was some decent stuff in the line, a lot of people agreed, but it was so galling to see it be so commercially focused.

And y'know what? We managed to bring out a LOT of books under that line, with a lot of good stuff in them. Because it SOLD. And if it sold on violent artwork and crunchy, market friendly powers, well, so be it.

You have to make compromises sometimes. Often it's a choice between putting out compromised art, doing some good, or being taken down, and doing no good.

The money HAS to come in. That's always true.

Funnily enough, I don't visit this site for game reviews. Ah well, whatever keeps you going.

And it's ALWAYS stars.

I prefer the Siskel and Ebert-style thumbs-up/down reviews. It tells me what I want better than a number ever could. It's also the reason why I read reviews on The Escapist as opposed to any other site: because you guys did it different, and did it better.

But now what? Are we just going to become another IGN or gamespy, morally questionable 'journalists' with a 9 point scale? That's a question that I want answered, and won't be answered until this new system settles in.

Dark Templar:

chantzzzzz:

Dark Templar:

chantzzzzz:

I do agree with the part about the people who read reviews only for a score are not likely to be reading reviews now anyway, so saying you might gain people is not far off the mark, actually.

So its a cash in. Sacrifice artistic integrity for higher traffic numbers. Thats how this is looking right now.

Just my opinion anyway. You guys review any way you want.

This isn't a "cash in". If you'd read about the part where Russ says that this might encourage people who only look at review scores to actually read the entire review, I'd say that that's looking more on the positive side of this change. Maybe people will actually realize that there's maybe something worth reading above those stars. And, as pointed out in the article, The Escapist (hopefully this is the right code to use!) is merely trying to keep up with trends, like online sites have to in order to maintain a presence in the online world.

I did read it, twice.

That part of the argument doesn't fly with me. I simply don't buy it as a viable reason for doing this. It just implies the possibility of some small positive side affect.

Get ready for whole bunch of "How can you give such and such game 3 stars??? What is the matter with you???" posts by the people who haven't read the review. Thus we see another kind of forum spam aside from the "FIRST!!!!111!!!!!11!1!1" posts. Do the possible positive outweigh the definite negative effects?

While this could be a concern, reviews already tend to be contested on a regular basis. Look at Zero Punctuation. No numerical score, just opinions, yet there are some pretty strong comments on some of those threads. People are never shy about disagreeing on other people's opinions, especially something as subjectively objective as a game review. Hopefully people will at least try to qualify their statements a bit more than just a brief opinion. In that case, there could be a lot more in-depth discussions about game reviews.

Sigh... such a shame.

One of the many reasons I always enjoyed the escapist was because of the lack of score reviews and possibly one of the many inspirations for my own reviewing.

While I understand your need I don't want the Escapist to end up as name in a list next to the Metacritic machine, we're so much better than that.

That's a real shame, I can never shake my cynicism that review scores exist only to sell games to people who don't like to read paragraphs. And also for them to be rewarded with having their name printed on the game's site and box. Cue a million threads arguing the arbitrary score of a new game also. The fact there are no scores is what really attracted me to this site. It was more personal and subjective. Call of Duty was good if you love fast paced shooters. S.t.a.l.k.e.r was good if you love slow paced survival shooters. What if the review prefers one over the other?
They may express it in words but the score will be lower. I understand that you'll say it changes nothing, but no matter what score is what a game is eventually judged by and when you do up the list page there is will be. When a friend watches a movie I don't ask him for a score I ask him if I'll like it.
It makes no personal difference to me but I just feel like what made this site so cool is being lost. Look at the right bar on the home page and you'll see what I mean.

I understand the need to make money and therefore make compromises, but I am afraid this decision will make the escapist 'one of the pack' and with that, lose its identity. Now that still makes money, but no fun.
In the end, the review will boil down to the score and the discussion will revolve around it. that is a sad internet fact I guess. People only interested in scores are now not part of your readers and will be in the future, but they might hurt your other user base. As you can see I am new here, I signed up in the end to comment on a GREAT article about lost love, depression and gaming as an escapist (pun intended) tendency.

I do not read or comment on zero punctuation, because it is impossible to filter through al the youtube like comments. I can already see myself skipping review comments. If I skip them, I think about the effect it'll have on the writer, and it'll surely be negative. I am afraid it will lead to less honest writing and more focussed on the scoring system. It will lead to degradation of quality, even if only because the lack of scoring is/was a blatant advertisement of intent I really liked...

Lastly, I think with the lack of scoring, but many great hits (like zero punctuation) on your site, you were educating some readers. probably not much, but still. Lured in by the fast-paced unique content, one such a user might actually read a review and find it refreshing and more informative. And thus another person is on the road of intellectual improvement. Educating your users isn't a bad thing, especially if it leads to a higher quality standard, but I understand it is not an effective (short term) business model.

In the end, the only good way to know if you like a game, is to ask people's opinion. But you need to understand where there opinion comes from, a scoring system is simply like asking "did you like it?" and stopping at "yes" or "no". It makes sense on a very broad scale. With many scores you can divide games in 'liked by nobody' and 'liked by everybody'. it is the games in between that might get left out, because apparently it is somewhat of an acquired taste and you need to see if it might be for you. It is in that category that games will fall that you will learn to love, because it is impossible to make a perfect game for everyone. For that reason it makes no sense for an individual site to score a game, the score is useless in itself.

 Pages PREV 1 2 3 4 NEXT

Reply to Thread

Log in or Register to Comment
Have an account? Login below:
With Facebook:Login With Facebook
or
Username:  
Password:  
  
Not registered? To sign up for an account with The Escapist:
Register With Facebook
Register With Facebook
or
Register for a free account here