If the abovementioned examples are anomalies in the typically challenge-oriented taxonomy of games, Gonzalo Frasca's September 12th is an anomaly of anomalies. As the introductory text puts it, "This is not a game. You can't win and you can't lose. This is a simulation." The simulation presents a Middle-Eastern city, complete with innocents going about their daily business, but the occasional terrorist lurks about. The player is offered one verb, one recurring choice: to shoot, or not.
While there is no explicit goal, there is an implied one: Use the only verb to eliminate all terrorists. It's not so easy. The simulation is tuned to provide a challenge against this implied goal, and the insurmountable nature of this challenge implies a political message. Every time a terrorist is assassinated, an innocent bystander will inevitably get in the way, leading mourners to become terrorists themselves. A positive-feedback loop kicks in, increasing the number of terrorists in direct relation to the player's active involvement. Though September 12th is a work of very low interactivity (one verb is about as low as you can go), it provides a very significant precedent.
Frasca has used challenge to make a statement outside of the system in which that challenge originated. In other words, challenge can be an artistic statement about the world at large, not just the game system.
Clearly, Frasca differs on at least one point with noted ludologist Espen Aarseth, who claimed games are, by nature, closed systems; culturally distinct entities with their own self-consistent logic. Aarseth's view is consistent with traditionally challenging games and the process of closed numerical tuning designers utilize to create and refine challenge. The result has been a sea of often entertaining, sometimes inspiring play experiences that, when the final boss finally crumbles, leave no lasting impression other than, "Hey, it's only a game."
Maybe we can do better. Maybe challenge can be used with social mechanics, not just abstract or physical ones. Maybe challenge can extend out of the flickering electronic box from which it's born and frame the player in ways never before considered. Maybe challenge can instigate cultural dialogues, inspire young people to better themselves, reflect light on unjust mechanisms within our society; maybe challenge can teach us something about ourselves.
But how? I'm not proposing we re-invent the wheel, it's likely the same principles which allow challenge to be created in closed, ludic systems can be effective in open, paidic systems, and anywhere between. In his paper, "A Preliminary Poetics for Interactive Drama and Games," Michael Mateas ascribes two features to interactive experiences in general: material and formal constraints.
Material constraints form the tools of play, what's given to the player within the closed, formal system, the "how" of accomplishing anything within the simulation. In Tetris, the falling blocks comprise the material constraints.
Formal constraints represent the ends of play, the rules which dictate or imply what the player should be doing, the "why" of play. In Tetris, the formal constraints include the torrential falling of blocks, the rule that a filled-in row removes all blocks from that row, as well as the scoring system and the speed-progression.
Mateas' paper lays out Quake's formal constraints: Everything that moves will try to kill you; you should try to kill everything; you should try to move through as many levels as possible.
Based on these formal rules, the player is given an arena with crystal clear intention. Creating challenge in this arena is simply a matter of providing just enough materials to keep the player alive, but on his toes. When the formal elements, the monsters and maze-like level designs, are pitted against a player running low on plasma ammo and resorting to a final cache of shotgun shells, this is when challenge is felt most palpably, when the opposing constraints put on the squeeze.
This principle of challenge, the squeeze effect, holds true for any sort of interactive experience. When the balance between formal and material elements is tweaked, the implications of the experience reverberate the loudest. People don't look back on their Quake play sessions and think in wonder of that time they jogged through a level, picking up the odd med-pack, thoughtlessly blasting monsters until the exit presented itself. People look back on that one deathmatch when their best friend had them pinned between an alcove and a chaingun spray, their armor quickly dwindling, and their teammate rushed in with explosive poise and gibbed the shooter with a rocket.