260: 1984 Out of 10

 Pages 1 2 3 NEXT
 

1984 Out of 10

A writer blasted reviews for over-hyping titles and giving too much credit to works that were just tripe. Peter Parrish proves that what George Orwell argued in 1936 for the novel is just as relevant to videogame reviews today.

Read Full Article

A very interesting article, sadly its not just the gaming industry affected by this kind of thing, but the whole entertainment industry trying to get exclusives for the next big thing... and they wont get that next exclusive unless project x from publisher/record company/studio y is given anything less than a stellar score

A few years ago i wrote Album Reviews for a Music site. We had a scoring system of 5 points with 0.5 Steps.

Even though the page never got famous enough to us having put up with commenters that would complain over if a 4 of 5 should be a 4.5, i deeply hated the scoring system (seriously, how was i even supposed to RATE MUSIC, one of the most subjective things in life with very few technical aspects that could be rated, in an arbitary "out of 5" system) and as an effect, hate ANY kind of scoring system.
It just does not work. You cannot rate games in a numerical system. If anything, it might give you an overall impression of what a game is like, but even then you might think differently about it. But i don't want to be redundant, this article covers the issue better than i could.

Hopefully, one day the gaming press overall will ditch thier nonsensical scoring systems altogether in hopes of finally becoming relevant.

The most important point was near the end. The fact people from multiple perspectives saw nothing wrong with succumbing to publisher pressure. I mean, whores typically brainwash themselves into believing their job is okay. That its not only their choice to sell themselves, but because of that illusion of choice, their job isn't degrading. Despite the fact if wasn't degrading, there wouldn't be prostitutes. As thats always been what they're selling.

If you live with something long enough it changes you. It isn't just publisher pressure thats the problem, its capitalism in general. Pursuit of profit has been slowly eroding all facets of journalism (and society) ever since people started believing money had any sort of value unto itself.

Also, the point about "thats like giving 50% just for writing your name correctly." Cuts both ways, really. A paper with no name (a game thats unplayable) is worthless. There aren't many in recent memory, so competence isn't exactly rewardable anymore, but like the fear2 5/10 review, you can't cut out the 5-point "name bonus" without looking like someone who's just posting negative reviews for attention. Which thanks to the internet's obsession with and phobia of trolling means you may as well have not reviewed at all, for as seriously as people would take it.

Of course, if someone releases a film that can't be watched, people look to the projectionist, not the studio.

pparrish:
There are, however, still enough robust individuals upholding these values to keep insightful games reviewing alive. Everyone reading this article will hopefully be able to name some favorites, and their continued work keeps gaming discourse above the level of advertorial guff.

Can anyone name some? I can't. I'd love to hear some if anyone has some suggestions.

I have yet to find a good, solid reviewer who I can trust. Just about anyone heaps praises, uses fallable scores or doesn't go into enough detail and analysis for me to warrant following them.

I have long since given up following review scores. It's a dire time in this industry where the games I most enjoy often come from small studios and receive mediocre score. However I find that you can often tell if the score is warranted by reading the reviewer's words carefully. Since most titles that they give a 8 or 9 out of 10 to leave them cold and uninspired (just like Orwell said), any title that really capture's their heart usually makes their writing a whole lot more jovial and sincere. "Wow, this is actually good and I've never played anything like this....i...is this fun I'm having? 12/10"

Reviews are almost worthless, not necessarily because of their commercial nature, but because of the straqnge values that modern publications use. Just like the OP said, games are rewarded for being competent and bug free. Games are rewarded for having 'replayability'. Games are rewarded for having 'multiplayer'. All this is done while the reviewer is claiming to be 'as objective as possible' and keeping a straight face when dishing out perfect scores.

What a load of bollocks. Why reward a game for being functional? Isn't that a requirement anyway? Why grant it extra points for replayability when the nature of some games just don't have it? I've played single player games for 3 hours that left a bigger impression on me as an artistic product than great, replayable games I've tried for 50 hours. Replayability should not be a critique point. Why try to be objectionable? You won't be anyway. I want reviews from people I can trust that accept and embrace their bias. Everyone has a favourite genre and previous games that captured your heart as a child, you should acknowledge that. I want raw reviews that aren't afraid to offend, because if they explain their score elloquently enough along with their justifications, no one should be offended.

It's quite amazing how spot-on Orwell was, even though he was talking about another medium in another part of history altogether. Just goes to show some things never change.

This is why I prefer to read reviews without scores attached to it. Or at least a review which scores each aspect of the game individually. I mean; the importance of different aspects of the game might differ for each person, and it's more difficult to understand a score when it has been heaped together.

This is one of the reasons why I gave up on IGN. In practice, their scores range only from 7.5 to 10, with 7.5 being given to only the worst games; it's a very broken and misleading system. It's also why I was a little dismayed when The Escapist started giving scores to the games they review, because I was worried that it would devalue the points they made in said reviews. This hasn't been the case for the most part, or at least I don't think it has, but what the future will bring remains to be seen.

Naturally, there are publications which assign scores to games which are worthwhile, and where the scores do mean something. I can't remember the magazine, but I recall a review which gave MGS4 8/10, after giving it a great deal of praise and calling it "the PS3's best game to date". The PC Gamer Presents range reflects this idea, with the byline, "It's a must if it's 80% plus". This is the correct attitude to have towards assigning scores to games, and indeed any medium.

I have to agree, there. A numerical scoring system can't realistically reflect a complex opinion. I was always bothered by ratings, possibly because they reminded me too much of school or something...

A simple numerical rating could be applicable to something purely objective, like how well a double back somersault is executed. Something we already know how it should be.

However, where innovation or originality are called for, there is always something new that comes up and people get over-excited about it like "oooh, that's a really cool feature, it's got to be the best game in the world"

Oh well, the article says it better than I do, anyway.

On the other hand, the trouble is that a rating system can come in handy for those who can't be bothered to read a full review. I just wish most people could just be honest in expressing their opinions.

That happened to me, on the consumer side of things, I used to read Gamespot's reviews before buying a game but they never had anything useful so I gave up, mostly I'll only play so-called 'AAA titles' from developers I already know because you can't find a review that really reflects what you want.

There is problem with reviews not being trusted but its not just limited to games, you still see the same thing with books, film and the art world in general. Those mediums have the advantage of longevity, there is a winnowing that takes place over time. All those overhyped but poor books disappear into the back of bookcases and forgotten. That film that won 7 Oscars ends up at the bottom of the dvd pile. The problem the gaming industry has that by the time that happens the game is so old fashioned hardly anyone plays it. Great games of the past like system shock2, halflife 1 and deus ex within 10 years or so wont even be playable.

Well, at least we have the reviewers on this site...

I feel lucky to have found this.

The way I see it there was one item left out of the reason for inflation, metrics. Large and vocal parts of the gaming community are obsessed with numbers; resolution, framerate, possible hours, actual hours, etc, etc.

When all this is going on, any score less than what they expect is a travesty, and any score more than they expect is propaganda and proof of bribery. When I really want to know how good a game is going to be for me, I do something insane and actually read the review.

Just remember, numbers are useless without context.

Interesting article. This is why I prefer reviews without a ratings system. I much prefer to read/watch reviews that just explains the good points and the bad points about the game and then I can just make my own mind up about whether to buy it or not.

This is a great article, Orwell impresses yet again. I find it funny that I was thinking about this just the other day, I was a huge fan of the first Mass Effect and was very excited for the sequel; which wholly disappointed. So many fundamental mechanics and design choices were different that I hardly recognized the classic I fell in love with. Yet it gets great reviews, just like the first. I wish I could keep fooling myself into liking it.

Props to Yahtzee for not falling prey to this concept, even if some people view him as an unappeasable ass because of it (not me, I'm just sayin')

Brotherofwill:
(...)

I have long since given up following review scores. It's a dire time in this industry where the games I most enjoy often come from small studios and receive mediocre score. However I find that you can often tell if the score is warranted by reading the reviewer's words carefully. Since most titles that they give a 8 or 9 out of 10 to leave them cold and uninspired (just like Orwell said), any title that really capture's their heart usually makes their writing a whole lot more jovial and sincere. "Wow, this is actually good and I've never played anything like this....i...is this fun I'm having? 12/10"
(...)
I want raw reviews that aren't afraid to offend, because if they explain their score elloquently enough along with their justifications, no one should be offended.

If it was just me, they could condemn those scores to hell. Scores are only useful if you have a list and want to compare stuff, but I don't see any reason for scores in a review as such. I'm not entirely sure but maybe this overhype is somehow related to the comparatively young age of video games as a medium. Correct me, but I think the first film reviews were also mildly euphemistic. Maybe we still have to learn how to write good reviews for games and not just put crosses next to our "graphics, gameplay, sound"-checklist. It's what I like about the weekly issues on the Escapist -- games are discussed more in socio-cultural terms and less in the pseudo-objective view of most reviews.

PR-pressure was the reason for me quitting my job as a game reviewer. New editor was more concerned about good publishers relations and less about constructive criticism.

Brotherofwill:
It's quite amazing how spot-on Orwell was, even though he was talking about another medium in another part of history altogether. Just goes to show some things never change.

If you consider how much video games borrow from other media (most prominently in terms of narrative, structure or cinematography), I don't find it too surprising. Same applied to the early movies, where most of the narrating voice-overs or the text implementation in silent films drew heavily from works of literature (besides, a lot of movies were/are adaptations of books).

I do find myself watching gametrailers.com video reviews, which do have meaningless scores; but the draw for me is seeing the footage of the reviewed game in action, which is quite valuable.

Jim Sterling of Destructoid seems to have realized this.

Giving Alpha Protocol a 2/10, Heavy Rain something low, too...And then giving Deadly Premonition a 10/10.

This is why I don't like rating systems at all. Some arbitrary number tells you almost nothing about the game itself and readers become more focused on that number than on anything else in the review. I can't count the number of times I've seen a comment along the lines of "How could they give this game an 8.9? It deserves and 9.0 for sure." As if 0.1 points is a grave insult to them and this game they likely haven't played yet.

You know, the Norwegian system for reviewing everything actually works pretty well. It is a scale from one to six, no decimals. I don't know if it is due to the size of the scale or more to the culture around it, but it is commonly accepted that the scale is:

1 = horrible
2 = bad
3 = mediocre
4 = Good
5 = Great
6 = Mindblowing

When I say mindblowing, it really needs to have something that is both new and awesome. Many reviewers gave Avatar a 5, meaning it was great, but not mindblowing.

And reviewers are good at using the whole scale, with very few movies/books etc getting 6 and about as few getting 1 and most of them getting 3 or 4.

A reason why no contries have adapted this though, is that for whatever insane reason, the scores are represented by dice. Between a review, there is a large dice with the score on it.

You all are so focused on the score aspect that you're missing the point of the article. Even without scores, we get hyperbolic praise for mediocre titles, which still doesn't let the system accurately describe games the reviewers genuinely like. My impression is that hype is more poisonous than regulatory capture for big games, as you can generally find reviewers that still aren't beholden to the latter (small blogs, usually), but it's terribly difficult to find people not unconsciously swayed by either hype or its backlash.

I do agree that score inflation is quite rampant among most mainstream magazines/sites like PC Gamer, EGM, Gamespot, and the like, but I'd really like to touch on the last point: that score inflation is partially due to publishers imposing their wills upon game critics.

I personally think that it is the job of every game company to do everything possible to put their games in the hands of consumers. If they write an e-mail requesting their game be given a positive score and list the merits, that's perfectly fine. What they're doing is making an argument towards one conclusion and being an advocate for their product. What is wrong is when game critics rely on the publisher's views instead of giving their own opinion. For instance, there's nothing wrong with saying, "Oblivion has a vast, open world that literally takes hours to explore," even though that might be right on the box, it's a truth and it's my opinion as well as that of the game publishers.

When advertising dollars get involved, it becomes a matter of the critic's integrity versus cashflow. The big example that you cited was the Gamespot Kane and Lynch scenario which does illustrate something very wrong with the way Gamespot does business. No reviewer should ever be fired because he did his job. His job is to criticize the game, separate the good from bad, and give an overall impression to the consumer. As long as he remains consistent in his feedback, there is no reason he should lose his job, regardless of what developers or fanboys might say.

WaderiAAA:
You know, the Norwegian system for reviewing everything actually works pretty well. It is a scale from one to six, no decimals. I don't know if it is due to the size of the scale or more to the culture around it, but it is commonly accepted that the scale is:

1 = horrible
2 = bad
3 = mediocre
4 = Good
5 = Great
6 = Mindblowing

I like this - simple and effective

But I'd rather reviews didn't give number scores and just described their opinion on the game/movie whatever with the positives and the negatives

Example:
+great acting
+well directed

-bad effects
-poor story

deth2munkies:

I personally think that it is the job of every game company to do everything possible to put their games in the hands of consumers. If they write an e-mail requesting their game be given a positive score and list the merits, that's perfectly fine. What they're doing is making an argument towards one conclusion and being an advocate for their product. What is wrong is when game critics rely on the publisher's views instead of giving their own opinion. For instance, there's nothing wrong with saying, "Oblivion has a vast, open world that literally takes hours to explore," even though that might be right on the box, it's a truth and it's my opinion as well as that of the game publishers.

I disagree with this. Why? Because a reviewer is supposed to give the consumer an idea of how much they'll enjoy the game, and even if a company listed their arguments to consumers as well as to reviewers, 99 out of 100 consumers wouldn't bother to read it. The game ought to speak for itself. If the game really is as good as the publisher would argue, than the reviewer should be able to list those exact positive points without having to read the letter first. If the reviewer isn't able to do that, then those points aren't valid.

I also wonder. Would it be okay if a reviewer had decided to give a game a 8.0 after playing it, but then read the letter and thought "hm, that is a good point" and gave it 8.5 instead? I mean, to me, if they changed the score at all due to the letter, then the score would no longer accurately display how much the reviewer enjoyed it.

WaderiAAA:

deth2munkies:

I personally think that it is the job of every game company to do everything possible to put their games in the hands of consumers. If they write an e-mail requesting their game be given a positive score and list the merits, that's perfectly fine. What they're doing is making an argument towards one conclusion and being an advocate for their product. What is wrong is when game critics rely on the publisher's views instead of giving their own opinion. For instance, there's nothing wrong with saying, "Oblivion has a vast, open world that literally takes hours to explore," even though that might be right on the box, it's a truth and it's my opinion as well as that of the game publishers.

I disagree with this. Why? Because a reviewer is supposed to give the consumer an idea of how much they'll enjoy the game, and even if a company listed their arguments to consumers as well as to reviewers, 99 out of 100 consumers wouldn't bother to read it. The game ought to speak for itself. If the game really is as good as the publisher would argue, than the reviewer should be able to list those exact positive points without having to read the letter first. If the reviewer isn't able to do that, then those points aren't valid.

I also wonder. Would it be okay if a reviewer had decided to give a game a 8.0 after playing it, but then read the letter and thought "hm, that is a good point" and gave it 8.5 instead? I mean, to me, if they changed the score at all due to the letter, then the score would no longer accurately display how much the reviewer enjoyed it.

deth2munkies:

WaderiAAA:

deth2munkies:

I personally think that it is the job of every game company to do everything possible to put their games in the hands of consumers. If they write an e-mail requesting their game be given a positive score and list the merits, that's perfectly fine. What they're doing is making an argument towards one conclusion and being an advocate for their product. What is wrong is when game critics rely on the publisher's views instead of giving their own opinion. For instance, there's nothing wrong with saying, "Oblivion has a vast, open world that literally takes hours to explore," even though that might be right on the box, it's a truth and it's my opinion as well as that of the game publishers.

I disagree with this. Why? Because a reviewer is supposed to give the consumer an idea of how much they'll enjoy the game, and even if a company listed their arguments to consumers as well as to reviewers, 99 out of 100 consumers wouldn't bother to read it. The game ought to speak for itself. If the game really is as good as the publisher would argue, than the reviewer should be able to list those exact positive points without having to read the letter first. If the reviewer isn't able to do that, then those points aren't valid.

I also wonder. Would it be okay if a reviewer had decided to give a game a 8.0 after playing it, but then read the letter and thought "hm, that is a good point" and gave it 8.5 instead? I mean, to me, if they changed the score at all due to the letter, then the score would no longer accurately display how much the reviewer enjoyed it.

I get that argument, but where does the line go. I think they shouldn't take the letter into account at all. If you agree with me on that, what would the point be in sending the letter in the first place?

As people like karhell and Frybird say, the current numerical scoring for games is almost completely useless! The only use it has is to see how the reviewer thought, it was; your personal opinion may be changed or altered just by reading the review before even buying the game, as the article points out.

For example a review may state that the story line was "unrealistic, uninteresting and dull." but your individual perspective will only be truly made up after playing or viewing the gameplay.

As this amazingly concise and precise article states: the hype that the Press and the reviewers give these games are also daftly absurd. What we need is proper reviews that are based on the individual appeal to a stereotyped group of people (oxymoronic and arguement invoking statement I know...) That people can then make their mind up.

Or perhaps, if people knew the reviewer's personal opinion boundaries; e.g. what quantifies a "bad plot" then that would be okay, as long as the reviewer was truthful and honest in his opinion and he wasn't just being abusingly pressured into providing a biased review, which we all know would never happen...

What a world we live in, eh?

At least we have the Escapist!

Anyway, I thought the article hit the nail on the head there, and I completely agree with it.

I don't really trust any scores/reviews at all anymore. I'm usually just going by my instinct and opinions of people I know. This has served me okay-ish so far.

I have to say, at the end of the day, it comes down to me, $5, and a rental store. I never completely trust large game reviewers. Its too easy for them to slap on a good score and call it a day. That being said I can't name any reliable reviewers. It seems everyone is shooting for the top to become big. Once they become big, they become complacent. Once they become complacent, they lose quality.

I've given up on reviews in general. I listen more to close friends on how much they enjoy a game, and if they do, I'll rent/borrow it for a day or so to try it. Orwell was right in that reviewers become more and more slanted to the top end of the spectrum (I've lost track of how many "Movies of the year" have been developed). He also has good ideas as to how the problems may be fixed. The problem is that the pressures will always be there. Should a reviewer become known for honesty, they will become more popular. As they become more popular, developers/PR people will lean on them more heavily. Thus, he/she will find themselves slipping to the inflated opinions.

YES. Very much yes. The situation has also made it hard to differentiate score-wise between a game in which the reveiwer simply unable to find enough wrong with to slap the "7/10" sticker on them without strange looks from his editor and a game which was extreamly decent but had a few bug and enough "Objectively" wrong with it to metit a downgrade to a 7/10. The reviewer may have found it an allround more inspired and better game albeit a little more rough but 7/10 is a 'safe' option.

Infact even 6s are getting more and more rare. and a common 5 or 4 is unheard of.

Scrumpmonkey:
YES. Very much yes. The situation has also made it hard to differentiate score-wise between a game in which the reveiwer simply unable to find enough wrong with to slap the "7/10" sticker on them without strange looks from his editor and a game which was extreamly decent but had a few bug and enough "Objectively" wrong with it to metit a downgrade to a 7/10. The reviewer may have found it an allround more inspired and better game albeit a little more rough but 7/10 is a 'safe' option.

Infact even 6s are getting more and more rare. and a common 5 or 4 is unheard of.

I never saw what the big deal was.

Just aslong as you're aware of it, you can translate 7 as mediocre, 8 as average, 9 as good, etc.
If this scheme fools the publisher, all the better. WE know.

The score inflation isn't a problem. The problem is lousy or dishonest reviewers and nothing will fix that.

veloper:

Scrumpmonkey:
YES. Very much yes. The situation has also made it hard to differentiate score-wise between a game in which the reveiwer simply unable to find enough wrong with to slap the "7/10" sticker on them without strange looks from his editor and a game which was extreamly decent but had a few bug and enough "Objectively" wrong with it to metit a downgrade to a 7/10. The reviewer may have found it an allround more inspired and better game albeit a little more rough but 7/10 is a 'safe' option.

Infact even 6s are getting more and more rare. and a common 5 or 4 is unheard of.

I never saw what the big deal was.

Just aslong as you're aware of it, you can translate 7 as mediocre, 8 as average, 9 as good, etc.
If this scheme fools the publisher, all the better. WE know.

The score inflation isn't a problem. The problem is lousy or dishonest reviewers and nothing will fix that.

Problem is, when is a 7 "a good 7" "A mediocore 7" or "I couldn't be aresed so here is a 7"

The consumer is left with a meanless score. A game like FEAR 2 does deserve a 5/10 because it is the very definition of bland and unremarkable. But because there was nothing game-breaking it got mainly 7s and 8s. Now take for example a more flawed but more abitious games like STALKER; Clear ~Sky. Utterly broken upon release but a great game in there that gets marked down for techincal issues to a 7/10.

some games are good but just don't engage enough or are two flawed/ held back to get a higher score than 7 BUT there is a whole raft of frankly boring shit that is stamped with the 7/10 because it runs nicely but nothing else.

Scrumpmonkey:

veloper:

Scrumpmonkey:
YES. Very much yes. The situation has also made it hard to differentiate score-wise between a game in which the reveiwer simply unable to find enough wrong with to slap the "7/10" sticker on them without strange looks from his editor and a game which was extreamly decent but had a few bug and enough "Objectively" wrong with it to metit a downgrade to a 7/10. The reviewer may have found it an allround more inspired and better game albeit a little more rough but 7/10 is a 'safe' option.

Infact even 6s are getting more and more rare. and a common 5 or 4 is unheard of.

I never saw what the big deal was.

Just aslong as you're aware of it, you can translate 7 as mediocre, 8 as average, 9 as good, etc.
If this scheme fools the publisher, all the better. WE know.

The score inflation isn't a problem. The problem is lousy or dishonest reviewers and nothing will fix that.

Problem is, when is a 7 "a good 7" "A mediocore 7" or "I couldn't be aresed so here is a 7"

The consumer is left with a meanless score. A game like FEAR 2 does deserve a 5/10 because it is the very definition of bland and unremarkable. But because there was nothing game-breaking it got mainly 7s and 8s. Now take for example a more flawed but more abitious games like STALKER; Clear ~Sky. Utterly broken upon release but a great game in there that gets marked down for techincal issues to a 7/10.

some games are good but just don't engage enough or are two flawed/ held back to get a higher score than 7 BUT there is a whole raft of frankly boring shit that is stamped with the 7/10 because it runs nicely but nothing else.

They have decimals for that. Consider a star system like it's being used here on the Escapist is about as useful as a 7 to 10 scale. The score alone won't allow for much comparison, because one star covers alot.

An honest and diligent reviewer if one exists, can in theory rank games in the same genre by score, with decimals (like 7.1 < 7.2). It would still be an opinion, but it would be quantifiable again.

Good article, I'd have to agree with a lot of it points. Personally I prefer to the avoid the whole problem myself by sticking to known variables like Valve, or Nintendo, which consistently make good games. Versus EA, which pushes a lot of average. If I do read reviews, I read multiple for the same game, and glean what useful information I can.

Interestingly enough, I find 1984 to be the single most over-hyped novel ever written. Sure it's a great book, but Animal Farm is pretty much the exact same thing, and I personally like it better. It is relevant yes, but these days people are way to deep into this whole "communism/socialism" thing. If you want to read a REALLY relevant novel, read "Brave New World" by Aldous Huxley, he pretty much predicts the future, then parodies it.

At the moment I'm too lazy to actually comment anything useful about this topic, but I nonetheless want to say the following: nice article, keep it up.

Yeah, that's all.

 Pages 1 2 3 NEXT

Reply to Thread

Log in or Register to Comment
Have an account? Login below:
With Facebook:Login With Facebook
or
Username:  
Password:  
  
Not registered? To sign up for an account with The Escapist:
Register With Facebook
Register With Facebook
or
Registered for a free account here