 Tennis for 2 is often cited as the first video game and it employed custom controls. It was a tennis game where you used a knob to control the angle of a ball and pushed a button to hit the ball over the net. Quaint as this interface seems, it was sufficiently compelling to draw the attention of most people at an exhibit, foreshadowing the invasion of play into the digital realm. The arcade era also employed custom controls for most of its games. Fighting games had their own stick and button format. Custom games had wheels, but when games became domesticated, we had a standardization of sorts. The NES controller's iconic design, facilitating 2D movement perfect for a game like Super Mario Bros. But computer games, with the mouse and keyboard, being better for shooters and strategy games. Once games entered the 3D realm, the limits of a d-pad were unveiled and required an analog stick, built into the design of the Nintendo 64 controller. So we go from bespoke to the universal. Controller designs both limiting and enabling certain kinds of games. Nowadays we even have motion controls that were prominent on the Wii and touchscreens that map screen space into action. Designers often harness new control methods for compelling new experiences, but more often than not there are gimmicky implementations that don't stand the test of time. In any case, controls are how we interface with the medium known as video games, the vehicle that connects us cybernetically with alternate worlds. It is interesting going back and seeing how people responded to different types of control methods. This famous review of Alien Resurrection reads, The game's control setup is its most terrifying element. The left analog stick moves you forward, back, and strafes right and left. While the right analog stick turns you and can be used to look up and down. Clearly we've gotten used to it, but that doesn't change the fact that playing a shooter on consoles affords less precision than on a mouse and keyboard. Console shooters then designed around this, enabling generous auto-aim to aid the player in overcoming their fear. Understanding why this is the case has to do with biomechanics and technology. Obviously the mouse and keyboard grants more precision and subtle movements. The thumb and d-pad can't microposition as effectively, but how can this be mapped conceptually? We can borrow terms from engineering, control theory and degrees of freedom, which speaks to how many dimensions an object can move and rotate it, the x, y, and z axis. In addition to this, we can add the insights of Steve Swink and his book Game Field. There is input sensitivity, versatility, and how combinatorial it is. So a controlled scheme that has more granular states recognized, like how you can walk by slightly nudging the analog stick, has more sensitivity, whereas a combinatorial scheme like a fighting game lets you hit buttons in conjunction for multiple combinations. He also conceives of the ASDR framework, attack, decay, sustain, release, which illustrates how quick an action responds, input lag, how long it is active or active frames, and how it releases once you let go. With this conceptual framework, we can see why the degrees of freedom and sensitivity of the mouse is better for aiming, and why fighting games can be almost entirely composed of buttons like on a hitbox. But this all begs the question, have our controllers frame the kinds of games that can be made, pushing us on a certain trajectory, or are controllers devised to facilitate certain kinds of play? When controls exemplify the second idea, standardized interfaces suggest the former. Designers innovate when a new technology presents itself, working within new hardware constraints. But are these constraints imposing degrees of freedom in more ways than one? Not just in the control we are granted, but how games themselves evolve? In his paper, A Theory of Gameplay Innovation, Stefan Bührer argues it's hard to find truly new game mechanics because we've locked ourselves into a cultural framework that restricts how we think about games. He illustrates this by mapping the relationship between player and game to five axes, directness of inputs, immediacy of inputs, the speed of the interaction loop, and the origin and destination of the shortcut of the interaction loop. He then challenges a few sacred cows of game design, the idea that they should be easy to learn and hard to master, where instead he says they should be easy to use, not learn. On top of this, the ideas of intuitive interfaces in UI and UX, simple isomorphisms between interface and game, informed by a culture of computation. Computational interfaces don't allow us to communicate context, they are rigid, and so we render them simple and transparent instead, bounded by procedural norms and the idea of intuitive design. Computers are not a partner in a dialogue or conversation, they are objects we do things to, and hence are simple input devices. In games, to communicate ideas, emotions, themes, we have to map buttons to certain things. To press F to pay respects, quick time events has a metaphor for certain actions. These are interpreted inputs, because the computer is frankly stupid. This leads to the predominance of what he calls action verbs, conventional mechanics, jump, shoot, move, etc. To break free of this, he advocates for the exploration of a new design space, using the aforementioned variables, facilitating new avenues of creativity. The theory is far more nuanced and complex than I can do justice you hear, involving information theory, cybernetics and feedback loops, among other things. For example, if we take just two variables, direct control like in action games, versus interpreted input like in a quick time event on the x-axis, and on the y-axis, immediate outcome versus delayed outcome, we can map where specific games lie. Arcade games are in the top right, because they have direct control and respond immediately. Conversely, text-based adventure games and strategy games are in the bottom left, because they involve interpreted input, language processing, and have a delayed outcome. What types of controllers enable these forms of design? For the arcade game, degrees of freedom for movement, but also button combinations for depth in action verbs. For computer games, the mouse were using the screen as a state selection mechanism, clicking on anything in a strategy game at will, and the keyboard, which allows for inputting language, which can be interpreted. Which came first, controller or design? I mean, you tell me. Much like the relationship between player and game, the one between controller and game is a feedback loop, a dynamic process that feeds into its own self-generation. It is often stated that Miyamoto guided the design of the N64 controller with that of Super Mario 64, a synergy between design and forms of control. Bira then explained something about the jump that is so simple yet revolutionary. I felt silly for not recognizing it before I read this. The jump action was a very abstract command. It depended on the character's state and on the stick position. Furthermore, when you pressed the jump button, you lost control of your character for a while. It was unheard of. The idea of associating physical commands with action verbs that provided slightly delayed outcomes took the industry by storm and led to a multitude of implementations, using the pump and dig dug, flapping your wings in joust, jumping while shooting in moon patrol. This leap led to three different strands. Games like Tetris, which had slightly delayed outcomes and more strategy. RPGs that were a blend of action on the aforementioned strategy and adventure games sitting in the middle, and interpreted inputs, like that we see in fighting games, and then quick-time events. We can go through the evolution of design space, but here is a map that Bira devised for where all current genres lie. We see why certain genres are more feasible on certain interfaces, which can limit what areas of design space are explored. This need not determine explicitly where the medium can go, but can wield a heavy influence. We can talk about certain design innovations, like Z-targeting, the lock-on, or the cover-based shooter, or even what motion controls allow us to express. We can look at the evolution of single genres like the skateboarding game, which goes from the interpreted and direct control of Tony Hawk with button presses for combos, to the more robust parallel between skateboarding human and skate, represented by using analog sticks for tricks instead. How do controllers enable new design? How designers adapt? This can be mapped by those interested using the conceptual tools provided. Game feel, control theory, innovation in conceptual space. But the idea of control I want to emphasize here isn't just about controllers or control theory, but how our design sensibilities and play preferences themselves might be under control. In his book Ludo Politics, Video Games Under Control, Liam Mitchell argues that games promise the fantasy of control to master a world and rule-bound procedural form using some interface. These, however, exemplify values of a society of control that extends beyond the game, as was spoken about by Gilles Deleuze in his essay on societies of control. Games, as a technology mediated by the illusion of freedom, algorithms and the cybernetic interface that is the controller and us, can help us see into these structures of control and even challenge them. Games that do this interrogate their own text reveal how we are governed by the illusion of freedom. Would you kindly in bio shock? You wanted to be a hero in spec ops, memes in Metal Gear Solid 2. Moreover, gamers can reject the procedural norms of the game. They can speedrun, hack, mod, create and program their own worlds to control. These ethics can then bleed into reality, a world to be hacked, brought under the dominion of the rebellious game. Do controls determine the evolution of games? Is itself a byproduct of a society of control where we valorize the role of technology in mediating, framing and configuring our futures? This is what I suspect we need to disavow if we want to think about design and aesthetic experiences in games that are not governed by a culture of computation, procedural norms, a society of control, technological fetishism and determinism. The game itself can help us see past the limits of its own medium by foregrounding this essential aspect of reality. Brothers A Tale of Two Suns has very strange controls, de-familiarizing even. You control two brothers simultaneously with two analog sticks, an absurdity at first glance. What this game reveals is both what can be conveyed by control and our practically infinite adaptability. The conflict of using these controls mirrors the conflict between the brothers as they search for a cure to heal their ailing father. Simple movement starts out cumbersome, but we eventually get the hang of it. The sense of mastery, control, is mapped onto the metaphor of connection. Moreover, the oft-heralded ending sequence of the game, which you can skip to hear to avoid. When one brother dies, the other can still invoke his abilities as if a watchful protector, a phantom of affection that transcends the corporeal in more ways than one. A phantom limb to the player, the memory of one's brother to an avatar. Control can be leveraged for aesthetics, self-reflection, art. In the infamous Edge Review of Doom in 1994, they concluded with this most glorious of memes. If only you could talk to these creatures, then perhaps you could try and make friends with them, form alliances. Now that would be interesting. Putting aside the silliness of reviewing a game, not for what it is, but what you want it to be, the essential argument here was not just for new kinds of design, but new forms of control. The reviewer wasn't taking into account the idea of what interpreted input could mean, or even if it was possible, given the interface and technology we were dealing with. But what he was actually asking for was new verbs, new interpretations, a non-spatial, non-direct action media. In his GDC talk, The Future of Video Game Stories, Jesse Schell talks about what he calls head verbs, actions that allow us to input new emotions for the computer to interpret. For example, facial recognition software that might use our face as a cue for storytelling, to sense emotion, or even voice recognition, the seeds of which we have seen in experimental games. He also talks about natural language processing, and then programs that might operate like a dungeon master, dynamically incorporating us into an unfolding narrative, pushing us perhaps towards the hollow depth. In essence, our design sensibilities can work in conjunction with technological innovations. The title of his talk, How Medium Shaped Story, suggests our history of controllers, design innovations and their interaction, has limited what we can and have expressed using the medium. But are we now perhaps not controlled by technology, but by fantasies? Given our understanding of the tight relationship between control and design, is this even possible if the technology doesn't enable it? This is part of a design space that hasn't even been conceptualized yet, not just the mechanics of immediacy and interpretation, but what those interpretations can map onto. What types of stories and experiences might be enabled by more nuanced emotional input, more intelligent computer interpretation, or more participatory storytelling? Ironically, I suspect it leads us back in time to the origins of storytelling itself, oral storytelling, collaborative storytelling. The kinds we presently do in multiplayer games perhaps, but with more sophisticated means of interacting with each other. Tennis for Two was a harbinger of a new digital age of interaction. Adventure was a text-based game that was converted into a spatial action game for the Atari home console for technical reasons. It converted text recognition into action verbs. Somewhere between the custom controls of Tennis for Two and Colossal Cave Adventure's concession to direct control lies an apt metaphor for why control has been conceptualized in a very narrow and particular way, as procedural, bound by direct action and the norms of computation. Breaking free of this not only requires we recognize this, but reconstitute our relationship to the machines that have framed our thinking in the first place, exercising control for once over ourselves.