World building is my favorite part of gamemastering, but (despite the countless hours I devote to it), world building cannot be counted as the primary or even secondary role of the gamemaster. A GM can be an amazing judge, storyteller, and adversary without ever taking up the art of world-building. In fact, the majority of gamemasters never create their own worlds or settings, instead relying on pre-established settings such as Forgotten Realms or Warhammer Universe. Pre-established settings became popular within the first generation of the tabletop gaming hobby, and they have only increased in popularity since then. Nevertheless, I believe that even if you don't intend to create your own world, an understanding of the art of world building can only deepen your skills as a gamemaster; much the same way taking film classes can help you appreciate film even if you don't ever direct one. Whether you are crafting your own sandbox or running D&D Encounters in 4e, there's much to gain by studying the art of world building.
Start with Rules and Genre
There's a certain school of thought that says that the setting of a game, the genre of a game, and the rules of a game can all be divorced from each other. This school argues that one set of rules is just about as good as another, and all that counts are how the GM weaves it together. In fact, this premise underlies the rise of the D20 Open Gaming License in the 1990s. I'm not part of that school! I believe they have to form an organic whole.
On this point, at least, I did some interesting research. As my thesis project at Harvard Law School, I conducted a study on how the design of 3 different massively multiplayer games affected the societies of those games. To cut short a 100-page research paper, the answer was "strongly". It turns out that every set of game mechanics carries with it certain implicit and explicit assumptions about how the world works. They are the physical laws of the game world. Just like the law in the real world affects our societies, the laws in the fantasy world do, too.
While the research was for MMOGs, tabletop examples abound. For instance, Cyberpunk 2020's Interlock System offers a 10% chance for any given roll to be a critical success and a 10% chance for a critical failure. Cyberpunk 2020 therefore implies a world where any punk can get lucky, and even the best of the best are eventually going to bite it from bad luck. If that's not how your world works, you don't want to use Cyberpunk 2020 to run it. Because what will happen is that punks will get lucky, and elites will die young.
Another example: Classic Traveller uses a fascinating character generation system in which players choose or are drafted into "careers" for their characters, such as army, marines, or merchants, and then spend anywhere from 4 to 20 years in service, graduating into play as seasoned experts. There's little to no character improvement thereafter. Classic Traveller therefore implies a world the opposite of the traditional "Heroic Myth" - age and experience trump youth. Young characters aren't potential heroes ready to unlock their potential; they are unskilled mooks. Luke Skywalker doesn't become a Jedi master, he dies. Classic Traveller also implies a world with lots of organized, institutional careers, diametrically opposed to the post-apocalyptic environment of most D&D campaigns. This is why so many efforts to use Traveller as D&D in space fail.