Microsoft: The

Microsoft: The "X" Men
The Perception Engineers

Spanner | 28 Nov 2006 11:04
Microsoft: The "X" Men - RSS 2.0

A previous report from the User Research Group highlights exactly how Tim and Dan's team deals with apparently minor aspects of a game (such as the user interface) to ensure even new players don't start off with a negative impression.

During tests of Combat Flight Simulator, the Group flagged a usability problem that anyone other than a trained psychologist might easily dismiss as nothing more than a niggle. One of the selections within a setup menu was a choice of three radio buttons for adjusting the A.I. level of computer-controlled enemy pilots. Although it was quite reasonably assumed most people would be familiar with the term "A.I." as an acronym for "artificial intelligence," an early study showed a combination of trifling factors conspired to confuse the users.

The main problem was with the term itself. Although the participants had been selected due to their gaming and flight sim experience, the term "A.I." was apparently not as well known as the development team had assumed. It may be common jargon for developers, but only two out of seven testers were actually familiar with it.

Had this minor problem not been highlighted, there was a danger novice users would begin their first few games of Combat Flight Simulator against an incredibly difficult enemy and scrap the entire game before getting to grips with it.

Dan confirmed this preemptive "tweaking" of a game (and its interface system) is what the Group is all about. "The idea behind the Games User Research Group is very simple: Collect unbiased data from real users during game development and use that data to make improvements in the game before it's released. Although the idea is a simple one, the process is far from simple." He laughs, summarily making light of his obvious hard graft. He continues:

"It's not as easy as just bringing in the target consumers and getting their feedback. Careful control and expertise in psychological research methodologies must be leveraged in order to ensure the information we get from users is unbiased. The nature of how the testing is set up, the interactions participants have with the experimenter and other participants all have to be carefully controlled in order to ensure the integrity of the data we collect.

"The reason our group consists of individuals with a strong background in psychology is that we are specifically trained to gather data from people in an unbiased fashion, and we are well aware of all the potential areas for bias to creep in and work diligently to minimize that possibility."

You'd think a team of psychologists searching and digging to find fundamental flaws in a game's design would make them pretty unpopular with programmers, designers and artists, and Dan can recall a time when the Games User Research Group certainly had to prove its worth to the rest of Microsoft Game Studios.

"Eight years ago, when the group was first forming, many development teams questioned the benefit of having psychologists conducting user research on their titles," Dan remembers, "but our group has grown substantially over the past few years, which speaks to the importance Microsoft places on the work we do.

"The most progress has been made in how teams approach making games. User Research is now seen as a key ingredient in game development, and the importance of our work is recognized by the development teams. Today, it's actively sought after by the teams."

Comments on