Grad Student's Algorithm Answers, "What is Indie?"

Grad Student's Algorithm Answers, "What is Indie?"

Our goal is
to use an expectation-maximization algorithm to learn the parameters of a model that we will then use to forecast, without
peeking at the tags, whether a given game is likely to be considered by Steam users as "indie", or not.

A recent Masters graduate from the school of Aerospace Engineering at Colorado University at Boulder decided to use his years of advanced maths training to answer the question that really matters: What is an indie game? The debate about how to define "indie" has been going on for years, and talking about it has put us no closer to a conclusive answer than we were before. Jackson Wagner developed a learning algorithm, fed it metrics about a whole slew of Steam games, like Tags, Scores, and Pricing. Eventually, the algorithm learned enough to be able to spot indie games about 2 of 3 times based on those metrics, having removed the "indie" tag, which is obviously cheating.

I'll spare you my hazy recollections of Gaussian mathematics, but if you're interested in the nitty gritty, the entire paper available for download here!

Generally speaking, it seems these metrics act like social demographics, or internet advertising, which takes some data about a person, compares that to the data of millions of others, and makes an educated guess about how that person will behave, based on empirical data. Just think of Steam stats as the browser cookies of game genres.

image

Permalink

The second something is labeled 'indie' is stops being so. Like calling something an untold story.

That is a pretty cool grad students project, but I wasn't aware there was such debate about what is indie or not...

How does "Indie" not mean "independant"? I always seen indie as a tag for a developer without publisher backing.

I don't really understand the project here.

Silentpony:
The second something is labeled 'indie' is stops being so. Like calling something an untold story.

Sounds like a fancy way of saying indie developers don't exist.

Zenja:
How does "Indie" not mean "independant"? I always seen indie as a tag for a developer without publisher backing.

I don't really understand the project here.

It's a little bit more loose than that.
Is Amnesia an indie game? That didn't have a publisher.
How about Warframe? No publisher.

Zenja:
How does "Indie" not mean "independant"? I always seen indie as a tag for a developer without publisher backing.

I don't really understand the project here.

The debate comes from people who, as far as I can tell, don't know that and try to divine what "indie" actually means. To them it's something like "a small scale game, made by a small team" and then they try to actually quantify what that means. Apparently, at some point a game stops being small enough to be independent. Or something.

Zenja:
How does "Indie" not mean "independant"? I always seen indie as a tag for a developer without publisher backing.

I don't really understand the project here.

Me neither.

"Indie" is shorthand for "independent". As in, a game that was developed and published[1] by the same development studio.

I've never understood where the confusion comes from.

Chaosian:

It's a little bit more loose than that.

To some, but why? Seems like a pointless convolution of the term.

Is Amnesia an indie game? That didn't have a publisher.

Yep.

How about Warframe? No publisher.

Yep. It is, too. As are games like Portal 2, Terraria, Dota 2, Grand Theft Auto V, and The Witcher 3[2]

DoPo:

The debate comes from people who, as far as I can tell, don't know that and try to divine what "indie" actually means. To them it's something like "a small scale game, made by a small team" and then they try to actually quantify what that means. Apparently, at some point a game stops being small enough to be independent. Or something.

Which begs the question: At what point do we draw the line between a 'triple-A' team and an 'indie' team?

What arbitrary number of team members do we choose as the divider? Would fifteen be considered 'indie' or 'triple-A'? What about thirty? Would a game with ten team members go from 'indie' to 'triple-A' if they suddenly hired two extra artists? What about non-dev-team members? Do we count translators, marketers, distributors, etc?

And that's not even going into the fact that, at times, not ALL members of a dev studio are involved in the game(s) the studio releases. Quite often the members of that studio are divided up into disparate groups, each of whom is working on a different project.

I'm baffled by the need some people have to overly complicate the definition of 'indie game'. It feels like some (those who wish to maintain their 'hipster' status by decrying the 'triple-A' industry) wish to draw some distinction between the games they like and those they don't.

[1] Marketing and distribution are different matters.
[2] These latter two are technically debatable, but the dev teams were part of the inhouse teams at Rockstar and CDProjekt, respectively. As such they can be viewed as 'independently developed'.

Vigormortis:
Which begs the question: At what point do we draw the line between a 'triple-A' team and an 'indie' team?

Honestly, I don't see why they need to be different, either. CD Project Red published The Witcher 3 themselves, so that game would be classed as "indie". It also has high production value and can be classed as AAA.

It's the term "AAA" that is way vaguer than indie, yet I hardly ever seen it discussed - most times I do, it's in a discussion about "what is indie" and it's put up as the polar opposite. Yet as far as I know about either of the terms, they aren't. "Indie" is just "independent" and "AAA" is "high production values". Nothing in those makes them definitely opposite. Sure, most of the time an independent dev does not have the resources to pour into a project to make it triple A but that doesn't mean you cannot have both.

Vigormortis:

Yep. It is, too. As are games like Portal 2, Terraria, Dota 2, Grand Theft Auto V, and The Witcher 3[1]

I know you have the footnote, but just to point out that Rockstar is a subsidiary of Take-Two Interactive Software, meaning that it isnt independant. Just for you to know, meh

[1] These latter two are technically debatable, but the dev teams were part of the inhouse teams at Rockstar and CDProjekt, respectively. As such they can be viewed as 'independently developed'.

Chaosian:

Zenja:
How does "Indie" not mean "independant"? I always seen indie as a tag for a developer without publisher backing.

I don't really understand the project here.

It's a little bit more loose than that.
Is Amnesia an indie game? That didn't have a publisher.
How about Warframe? No publisher.

The reason there's still so many things wrong with League of Legends is because Riot Games is a small indie company, please understand.

Congratulations, Mr. Wagner! You created an algorithm to predict how current Steam users react to a game name 2 out of 3 times! This surely won't be abused at all... /s

So...I decided to have a look at the paper and I would say it probably isn't entirely serious.

From the actual paper:
I've been having fun with the gag of trying to be scientific about a ridiculous videogame-related debate

Methinks, a joke is getting a bit more publicity than it probably deserves.

Vigormortis:
Yep. It is, too. As are games like Portal 2, Terraria, Dota 2, Grand Theft Auto V, and The Witcher 3 (These latter two are technically debatable, but the dev teams were part of the inhouse teams at Rockstar and CDProjekt, respectively. As such they can be viewed as 'independently developed'.)

GTAV and W3 are about as independent as any given Ubisoft, EA or Activision developed game. Any claim of being indie flies out the window when divisions and subsidiaries come into play as descriptors.

Shit, GTA is the least indie thing possible, once you look at the entire Take-Two pyramid. And the funny thing about Witcher is that it was never indie, as CDP started out as a distributor and publisher in Poland, CDPRed, their development division, came a lot later. (unless you count poverty as being indie, which is the exact opposite of your sentiment)

DoPo:

Honestly, I don't see why they need to be different, either. CD Project Red published The Witcher 3 themselves, so that game would be classed as "indie". It also has high production value and can be classed as AAA.

It's the term "AAA" that is way vaguer than indie, yet I hardly ever seen it discussed - most times I do, it's in a discussion about "what is indie" and it's put up as the polar opposite. Yet as far as I know about either of the terms, they aren't. "Indie" is just "independent" and "AAA" is "high production values". Nothing in those makes them definitely opposite. Sure, most of the time an independent dev does not have the resources to pour into a project to make it triple A but that doesn't mean you cannot have both.

Which is kind of my point.

The budget for Dota 2 or League of Legends are colossal, yet they are, by definition, 'indie' games.

I feel that if people want to differentiate between big budget games and small budget games they should find better terminology than 'triple-A' and 'indie'.

josemlopes:

I know you have the footnote, but just to point out that Rockstar is a subsidiary of Take-Two Interactive Software, meaning that it isnt independant. Just for you to know, meh

Right, but check who actually published the game. https://en.wikipedia.org/wiki/Grand_Theft_Auto_V

Rockstar did it themselves. Take-Two took a back seat on that one.

MC1980:

GTAV and W3 are about as independent as any given Ubisoft, EA or Activision developed game. Any claim of being indie flies out the window when divisions and subsidiaries come into play as descriptors.

Do they, though?

I'll get back to this, but first...

Shit, GTA is the least indie thing possible, once you look at the entire Take-Two pyramid.

See the link just above. Rockstar was the publisher.

And the funny thing about Witcher is that it was never indie, as CDP started out as a distributor and publisher in Poland, CDPRed, their development division, came a lot later. (unless you count poverty as being indie, which is the exact opposite of your sentiment)

Only a portion of the entire team of designers at Re-Logic are working on Terraria 2, while the rest are split between updates for Terraria and development of Terraria: Otherworld. Do we suddenly consider those to no longer be indie games?

If not, why then do we consider, say, The Witcher to not be independently developed and published, just because only a portion of CDProjekt is making it?

Seems like you're arguing more for a division between 'indie' and 'triple-A' based on team size or budget. In which case, League of Legends isn't indie while something like Portal is.

This is really cool article! Escapist needs more of news like that :)

And on OP, this student doesn't answer what is or isn't 'indie'. He just compares how well simple data mining algorithm fares in assigning 'indie' tag to a game given its: price, count of owners, count of users which run it last 2 weeks, time spent on game and avg. length of play session. Results were well medicore (66%) but lets be honest given available data quite nice (above random).

I wonder how xgboost would fare here.

BTW Valve probably uses 'similar' approach to determine which games you would like in your featured and discovery queue.

Personally, I think "indie" has evolved to be a term completely different from just being shorthand for independent.

At this point, as people have stated, it generally means a smaller-scale game made by a smaller-scale team, though not neccesarily self-published. Of course, with how hard it can be to define the line of what is and isn't "small scale" I think it tends to fall on general community consensus whether the game is "indie" or not.

Worth noting on the prospect that a game doesn't need to be self-published to be "indie", I'd like to point you towards Devolver Digital, who are flat-out considered an "indie game publisher". That'd be pretty impossible if the main metric for indie was being self-published. Top that off with the fact that if you ask someone to name some big indie games from the past few years, you've got a strong chance that Hotline Miami and Broforce (both published by Devolver) will both be on that list, and it becomes pretty clear the term has evolved beyond simple shorthand.

(Point of note, of course, is that Devolver and other "indie publishers" like KiSS are generally 100% hands-off with the development process of games they publish, just taking care of promotion/distribution, which is a big part of what IDs them as an "indie pub" in the first place.)

(Also don't even get me STARTED on how we now have "AA Games" as its own subset to fill in the vauge area between indie and AAA.)

Vigormortis:

MC1980:

GTAV and W3 are about as independent as any given Ubisoft, EA or Activision developed game. Any claim of being indie flies out the window when divisions and subsidiaries come into play as descriptors.

Do they, though?

I'll get back to this, but first...

Shit, GTA is the least indie thing possible, once you look at the entire Take-Two pyramid.

See the link just above. Rockstar was the publisher.

And the funny thing about Witcher is that it was never indie, as CDP started out as a distributor and publisher in Poland, CDPRed, their development division, came a lot later. (unless you count poverty as being indie, which is the exact opposite of your sentiment)

Only a portion of the entire team of designers at Re-Logic are working on Terraria 2, while the rest are split between updates for Terraria and development of Terraria: Otherworld. Do we suddenly consider those to no longer be indie games?

If not, why then do we consider, say, The Witcher to not be independently developed and published, just because only a portion of CDProjekt is making it?

Seems like you're arguing more for a division between 'indie' and 'triple-A' based on team size or budget. In which case, League of Legends isn't indie while something like Portal is.

Yes, the claim those 2 games are indie is ludicrous. By your logic, and this is not hyperbole, games like Assassin's Creed and Overwatch are 'indie'. That is simply asinine.

"Rockstar published it", how does that validate anything you said? EA publishes games by EA, oh how indie of them.

Rockstar 'Games' published it, which is a subsidiary of Take-Two, and Rockstar 'North' developed it, along with half a dozen other studios the world over, all of which are subsidiaries of Rockstar 'Games'. Not one iota of that structure could be misconstrued as 'independent', 'cause Rockstar sure as shit ain't. Replace Rockstar with Ubisoft and North with Montreal, what do you get? Exactly. This all can be found in that wikipedia page you used. Why are you linking something, if you don't even bother to understand the contents?

Terraria has nothing to do with anything, and is an incredibly poor comparison to CDP. Flimsy parallels aren't making your argument stronger. You seem to be really hung up on 'portion' this, 'in-house' that, while completely ignoring the companies' structure and history, the thing that actually decides whether something is independent or not.

CDProject Red, developer of the Witcher games, is a division created in 2002 to develop and port games by the polish distributor, publisher and localiser CDProject, founded in 1994. These are facts, and no matter the initial poverty of CDP, this makes Witcher not an independent game. The end.

And where in the name of blue fuck did you get that I was saying that 'team size' and 'budget' are the separator between indie and otherwise? Would you kindly reply to the contents of my comment next time, and not the interpretation you want to argue.

MC1980:
And where in the name of blue fuck did you get that I was saying that 'team size' and 'budget' are the separator between indie and otherwise?

Maybe because you complained about team size and budget.

MC1980:
Would you kindly reply to the contents of my comment next time, and not the interpretation you want to argue.

Actually, you know what - how about you actually decide to make a point first. Because otherwise all interaction will devolve to you trying hard to find flaws in what was said. Seen it a thousand times, so let's try to break the cycle early - how about you describe what indie is and why that is, instead of just trying to poke at other explanations.

Because this is why there is such a "debate" - a bunch of people just go "no, it's not this" without actually offering any alternatives. It is not that people can't agree on what is indie but it's like an endless argument on what isn't.

DoPo:

MC1980:
And where in the name of blue fuck did you get that I was saying that 'team size' and 'budget' are the separator between indie and otherwise?

Maybe because you complained about team size and budget.

Maybe read my original comment? Here let me quote it for you:

Look at all those complaints about budget and team sizes! Cleary the word 'division' means lots of moolah. And 'subsidiary' is codeword for people by the fuckloads. Get the boat.

You'd think, that on a forum, a medium where the primary method of communication is through writing, people would know to read and respond to what's written, and not what they conjure up in their minds. Why are you even talking with people, when you make up bullshit to put into their mouths and then respond to that? You don't need 2 people for that masturbatory imitation of a conversation.

DoPo:

MC1980:
Would you kindly reply to the contents of my comment next time, and not the interpretation you want to argue.

Actually, you know what - how about you actually decide to make a point first. Because otherwise all interaction will devolve to you trying hard to find flaws in what was said. Seen it a thousand times, so let's try to break the cycle early - how about you describe what indie is and why that is, instead of just trying to poke at other explanations.

Because this is why there is such a "debate" - a bunch of people just go "no, it's not this" without actually offering any alternatives. It is not that people can't agree on what is indie but it's like an endless argument on what isn't.

How's about you realise the nonsense you just wrote? See guy, just because you say "eeuh, you didn't make a point!?!?!" doesn't mean that that's true. Actual, basic reading comprehension. Guy said some shite, I responded it was shite, he responded with more shite, I respondend with shite disproving his shite. Points were made, whether they were conjecture on his side or disproval on mine. So that whole holier than thou bullshit about me not making points was hogwash from the word go, that I'm sure you found really smart in the heat of the moment. Happens to the best of us.

Second, how's about you not try to steer the conversation in a place where it was never intended to begin with? 'Cus you see fam', I was responding to the specific conjecture brought forward by one Vigormortis, specifically pointing out a part of his spiel as mad nonsense. That was the extent of my interest and desire to participate. So who the hell are you to barge in, pretend nothing I wrote was of value, despite them being more than fair counterpoints, and then demand I respond to something I don't give a fuck about, after having the contents of my previous comments be ignored in favour of your posturing?

MC1980:
respond to something I don't give a fuck about

And yet here you are. If you really didn't give a fuck about it, why respond at all. And why are you trying hard to "posture". Funnily the same thing you're accusing me of.

You still don't seem to have a point, just trying to disagree with other people.

Chaosian:

Silentpony:
The second something is labeled 'indie' is stops being so. Like calling something an untold story.

Sounds like a fancy way of saying indie developers don't exist.

Zenja:
How does "Indie" not mean "independant"? I always seen indie as a tag for a developer without publisher backing.

I don't really understand the project here.

It's a little bit more loose than that.
Is Amnesia an indie game? That didn't have a publisher.
How about Warframe? No publisher.

Then yes. Just because the game has good funding doesnt make it not independant.

MC1980:
You accused me contrarianism, yet here you are, doing nothing to respond with intellect, fishing for portions of sentences you can latch onto, without regard for context or meaning.

It's because you're trying to lead everything astray. You did not respond to what I asked, so I see no reason to respond to what you are saying. So do you actually want a proper conversation or not? Because so far you haven't tried to have one.

MC1980:
So let me break it down into child's speak:

"don't give a fuck"->[referring to]->your inane demand that I give you a definition for a word

It's the core of what is being discussed here. Without it, the discussion is basically meaningless. You will keep going "No, t's not that. No, it's not that, either." which is not very productive.

MC1980:
You barged in playing the arbiter, claiming savvy, yet all you've amounted to is bluster. Some arguer you are.

I didn't "barge in" anywhere - it's an open public forum. Something you already noted yourself. If you didn't want to speak to anybody else, you can take your conversation to private messages. If you want a public discussion, you aren't very good at facilitating one.

MC1980:

Yes, the claim those 2 games are indie is ludicrous.

How so? You keep SAYING it is but never offer any explanation as to WHY.

Maybe start with explaining your definition of 'indie' instead of just hurling ad-hominems at me? Thanks.

By your logic, and this is not hyperbole, games like Assassin's Creed and Overwatch are 'indie'.

How do you parse that? Assassin's Creed isn't indie because the developer and publisher are two different companies, two different teams. Overwatch, however, could be considered indie. It's made by and published by the same company, the same team of people.

But since you consider that "asinine", then you clearly have a different definition of 'indie'. I'm only aware of a few other definitions for the term, two of which are divisions based on team size and/or budget. So, until you actually bother to clarify your position, I have no choice but to assume you hold to one of the definitions I'm familiar with.

"Rockstar published it", how does that validate anything you said?

Because my definition of 'indie' is a game developed and published by the same team within a company. If Rockstar coded the game and later published it then, by definition, it is 'indie'.

Again, if you disagree, perhaps consider sharing YOUR definition instead of being rude and just insulting me.

EA publishes games by EA, oh how indie of them.

In your odd, and unwarranted, fit of anger, you've completely missed my point.

EA has no real 'in-house' dev teams. They have contracted subsidiaries and 2nd party teams housed in other companies. So, for example, Mass Effect is not 'indie'. The development team and the publishing team are completely separate, each housed in a different company.

I don't understand why you're having so much trouble understanding my definition. Maybe it's a lack of information on certain games? I dunno...try Wikipedia, maybe?

Rockstar 'Games' published it, which is a subsidiary of Take-Two, and Rockstar 'North' developed it, along with half a dozen other studios the world over, all of which are subsidiaries of Rockstar 'Games'. Not one iota of that structure could be misconstrued as 'independent', 'cause Rockstar sure as shit ain't.

But Take-Two didn't publish the game, Rockstar did.

And, more over, those other teams are still a part of Rockstar. Do you consider, say, Star Citizen to not be indie, even though it's being developed by numerous, disparate companies? Would you consider Minecraft to not be indie, even though it too has had more teams involved than just those at Mojang?

Hate to beat a dead horse, but it would REALLY help if you got around to explaining your position. The insults and overly hostile tone aren't really helping.

Replace Rockstar with Ubisoft and North with Montreal, what do you get? Exactly.

Not really equivalent, but at this point I'm not sure I even could get my point across to you. You've become so hostile, over fucking nothing, that it feels like you're only interested in shouting at me rather than have a discussion.

This all can be found in that wikipedia page you used. Why are you linking something, if you don't even bother to understand the contents?

You're right. One of us certainly didn't understand the contents of the wiki page...

Terraria has nothing to do with anything,

Why not? It is, by most definitions, considered an 'indie' game. It's also widely popular. How does its inclusion into the discussion not merit a fair comparison?

and is an incredibly poor comparison to CDP.

How so? We're discussing what constitutes a game being considered 'indie'. If part of the definition includes things like budgets, team sizes, popularity, etc, how are Re-Logic and Terraria not a fair comparison?

Flimsy parallels aren't making your argument stronger.

At least I have a position. You've spent your entire diatribe on insulting me and other posters over our positions, while failing to offer your own.

You seem to be really hung up on 'portion' this, 'in-house' that, while completely ignoring the companies' structure and history,

No, that seems to be your issue. I'm specifically looking at the company structures when I gauge team compositions and a games status.

the thing that actually decides whether something is independent or not.

So your definition IS based on things like team size, budget, and team structure? Why, then, did you attack me for implying as much?

That's just bafflingly rude, dude. Seriously.

CDProject Red, developer of the Witcher games, is a division created in 2002 to develop and port games by the polish distributor, publisher and localiser CDProject, founded in 1994. These are facts, and no matter the initial poverty of CDP, this makes Witcher not an independent game. The end.

Wait, so now your definition is based on budget? I'm confused.

Please pick a definition and stick to it. This waffling back and forth is maddening and confusing.

And you said it yourself. CDPR is an in-house division of CDP. Ergo, a dev team WITHIN THE COMPANY. Not some separate entity. Not some contracted subsidiary. An in-house team. That, by definition (including one of your own) makes the The Witcher an 'indie' game.

You really need to define your position. As it stands you contradict yourself from moment to moment. One minute you're using a definition that could define one game as 'indie', the next that game suddenly becomes not 'indie'.

And where in the name of blue fuck did you get that I was saying that 'team size' and 'budget' are the separator between indie and otherwise?

Among other things, from comments like this:

"You seem to be really hung up on 'portion' this, 'in-house' that, while completely ignoring the companies' structure and history, the thing that actually decides whether something is independent or not."

"These are facts, and no matter the initial poverty of CDP, this makes Witcher not an independent game."

These imply divisions based on budget, team size/composition, etc. If they don't, please elaborate because otherwise your position is strange and contradictory.

Would you kindly reply to the contents of my comment next time, and not the interpretation you want to argue.

The irony is palpable.

DoPo:

MC1980:
respond to something I don't give a fuck about

And yet here you are. If you really didn't give a fuck about it, why respond at all. And why are you trying hard to "posture". Funnily the same thing you're accusing me of.

He certainly seems quite livid over something he cares nothing about.

You still don't seem to have a point, just trying to disagree with other people.

Maybe that IS his point...

 

Reply to Thread

Log in or Register to Comment
Have an account? Login below:
With Facebook:Login With Facebook
or
Username:  
Password:  
  
Not registered? To sign up for an account with The Escapist:
Register With Facebook
Register With Facebook
or
Register for a free account here