Michael Russell, the quality assurance manager at Ritual, detailed his experiences with Sony’s Q&A department after being offered a job at the company.
After Sony flew in the former lead tester from Microsoft Games Studio first class for a job interview back in April, Michael Russell claims he turned down the job due to Sony’s general Q&A practices.
“The straw that broke the camel’s back came in the last hour of my interview. I was told that the way that Sony tests their games is that there are one or two test leads on a project starting at about six months out. At T-8 weeks, between 80 and 100 temporary testers are brought on to test the game for those eight weeks. That’s it. This was done for financial reasons, and as a QA Manager, I would be expected to run test the same way.”
Russel disagrees with this approach, citing the greater chances of game delays or release day bugs when compared to having more testers in place from the beginning of development. Aside from the the technical issues, Russel also didn’t like the general attitude Sony seemed to have toward its game testers.
“There was talk about issues that only came up on production UMD’s for PSP games, major friction between test and development teams with little to no management backing for test, little to no shared technology, extremely lax “user effect” bug metrics for determining whether or not to fix something, and a variety of other fairly hefty issues, not just from a process standpoint, but a overall culture standpoint,” said Russel. “Microsoft is known for giving QA a bit too much say in the products that are developed, but the feeling I got inside Sony was that QA was seen as nothing but a bunch of monkeys with controllers.”
This blog post was sparked by Sam Kalman, another game tester, who discussed the reproducible bugs found in the Sony-published Genji for the PS3. The first is an issue that prevents the player from finishing a quest if he tries the wrong solution to a puzzle.
“I’m 100% sure that this bug was a known issue when the decision to ship was made,” said Kalman. “It’s more likely that the stakeholders decided that the bug was not severe enough to fix in the remaining time prior to ship. Someone might have performed a calculation comparing the estimated number of people that this bug would affect and the cost of money and/or risk involved in fixing the bug for the final game.”
Kalman went on to detail bugs that prevent the player from finishing the game at all: “Of course, there is no way for me to be sure what happened during testing, or why this game did ship with what I would consider to be a severity 1 bug. But I do know that this game did ship with a blocking issue. You can’t finish the damn game because of this bug, and you have to start over from the beginning unless you were lucky like Chris. In my mind, this is worse than a crash bug, and right up there with data loss bugs. I think it’s inexcusable that a game of this profile would ship with this type of issue, and I encourage anyone affected by this bug to scream bloody murder to Sony Support.”
