On Wednesday Greg Tito had a bit on the matchmaking in StarCraft II and how designer Rob Pardo is considering making the system "more inaccurate" when finding you an opponent. Currently, you ask Battle.net to find you a match, and it will seek out someone at your skill level and put you in a match with them. Pardo is suggesting the system should have some randomness so that occasionally you'll face people a little above or below you.
The problem they're up against is a big one. The average single player game divides players into the three tiers of "easy, normal, hard". But in a complex competitive game against human beings, the delta between the levels of players can be massive. Newbie Allan won't be able to win more than one in ten matches against casual player Betty. Betty will have a similar result against Carl. Carl won't do better than one in ten against Expert Donna. Donna will have the same level of challenge against pro-level Evan. There are many orders of magnitude between the top and the bottom, and the game needs to let everyone find a game that they'll enjoy. It isn't fun to lose all the time. But the same goes for winning.
Back in 2002 or so I played competitive Unreal Tournament on an almost daily basis. That game had a global ranking system that kept track of not only your kills-to-death ratio, but also tracked the skill levels of your victims/killers. Killing a highly-ranked player would give you a nice boost to your ranking, while slaughtering players far below you would give you almost no benefit. Conversely, being killed by a top player wouldn't hurt your standing nearly as much as being killed by someone below you. A system like this makes it very, very hard for people to reach and hold the topmost positions. If you want to hold one of those "top 100 players" slots, you don't just need to play well, you need to routinely play well against other high-level players. It also gives you an incentive to avoid beating up on players below you, since you have a lot to lose and almost nothing to gain from the encounter. The system rewarded players that sought challenge and discouraged people from just pummeling newbies.
But the real beauty of the system was that you had control over where you played and who your opponents were. I found a server with some very highly ranked players on it, and made that place my home. It was brutal at first, with me sometimes scoring in the single digits while the top players would be in the triple digits. But I stuck with it, and after a couple of weeks I could see my scores and my ranking slowly improve. I was becoming a better player by pitting myself against far superior enemies. When I reached the point where I was occasionally topping the charts on that server, I knew I'd come a long way and earned my place. It was a tough climb, but it was something I was free to choose to do. I could have chosen a more gradual but less punishing path by seeking out players of my own level. Once in a while when I got frustrated, I'd do that.
I think it's a mistake to create a system like the one proposed on Battle.net where you can only play against people of your own skill level. Over time, you'll always win 50% of your matches. If you start to improve, the game will just find you harder opponents. No matter how good you get, you'll still have basically the same experience. There won't be easy or hard games. Pardo is thinking of making the system more "fuzzy", so that you'll occasionally face someone a few notches above or below you. That will make the game less monotonous, but I don't think it will lead to more satisfying matches. It means a lot more to finally overcome a superior player who has beaten me the last ten games in a row than it does to beat some random name that's ranked a few points above me, even if the two outcomes are statistically the same.