 And we are streaming this. So if you tweet at hashtag ML Talks, we will, if you ask interesting questions, we'll raise those questions later. So the program, first of all, it's John Sharpe and Colleen McQueen. And they will be talking about perspectives on creativity and failure and their new book that happened to come out at the time that they were preparing for this talk. And we'll have a conversation and we'll open it up. And so we'll start out with a presentation from them. Colleen and John. Should I sit next to you? I guess I don't know. Where do you want to sit? I like sit next to you. I will apologize in advance for this foot. I'm not used to sitting while I'm talking, so it's probably going to start doing this that I won't even notice. How do we make slides go up? Thank you. So as you can tell, one of them's John and one of them's me, but our hair has changed since then, and now we kind of look the same. So we can't tell us apart, right? My voice is a little deeper. Otherwise, I think we're about the same. All right, let's start. Yeah, I think today may be the day the book officially exists in the world. Yeah, today is the day the book is out. You can find it at the MIT bookstore. Not too far from here. We saw it this morning. So a little bit about us. So John and I both teach at the new school, specifically at Parsons School of Design, which is a college within the new school. We also run a lab called Pet Lab. We love Labrador Retrievers, and I had a pet lab for a while, but it stands for Prototyping Education, Technology, and Learning Lab. Abs. And there we do a lot of different projects. We mostly focus on play and learning and games, and we're working with all kinds of partners like the Red Cross, the US Holocaust Memorial Museum, and others. Juck some water. Thanks. We also work together with a colleague, Eric Zimmerman, and I guess it's a company, Local Number 12. Two of our games are pictured here. The Metagame, which is the daily show, which will save the world in the Big Mac. It's in that Cards Against Humanity slash Apples to Apples variation. We think of it if Cards Against Humanity is junior high school humor, we're grad school humor. Is that how we like to think of it? And then we have a game coming out this fall called The Lost Words. We take public domain literature, and we make word puzzles out of it. So look for that somewhere on the little tiny computer in your pocket this coming fall. And then finally, this is our second book, Iterate. It follows games, design, and play, a detailed approach to iterative game design. We're going to try to figure iteration into all of our books, I guess. But that's really where we're coming from. We're game designers. So failure is the topic we're supposed to talk about today. I used to live in Atlanta. I think a couple of people in the crowd used to live in Atlanta. You may remember the story where a big chunk of the raised highway just collapsed about a year or so ago. Well, this isn't the kind of failure we're talking about. We're not talking about the catastrophic kinds, the kinds that mess up lives, end lives, complicate things. Instead, we're talking about creativity. So what we're really focusing on is how you go from idea to actual thing and the failures along the way. And we're ultimately interested in how different creative practitioners manage and deal with failure, how they think about it, what their perspective is. We've done a lot of different kinds of things between us. We've been photographers. We've done some filmmaking, game designers. You were a DJ. I was a DJ. We have more websites that we can count. We've been involved in all sorts of mediums. But most recently, game design for about the last decade is what we've been up to. And we know how game designers deal with failure, but we're like, well, we've seen it in other disciplines. But it led us to like, well, who else can we talk to? And who else can we learn from to help us better explore this? Henry Petrosky, I think, is the muse or patron saint or something of this project for us. This quote was something we bumped into pretty early in our work on this project, which I guess we should admit has been going on for seven years. A lot of false starts, a lot of failures. We started on this project when Colleen was on sabbatical. She's on sabbatical again. So it took us an entire professor's academic sabbatical cycle to actually make this thing come to pass. But this quote from Petrosky really resonates with both of us that failure can be a good thing if you know how to deal with it. Ultimately, we really think of, and after we talked to a lot of different practitioners, we began to think of failure as an illumination, as a flashlight, right? If you've ever encountered that classic problem where you push the door, but it's supposed to be pulled, and it says pull on the door, have you done this? I think we've all done this. All of a sudden, how that door works, that door handle, everything about that door clearly comes into focus. If we hadn't accidentally pushed when we were supposed to pull, we wouldn't have noticed those mechanisms at all. So failure is like a flashlight. It reveals how things work to us. It provides us with clues. And hopefully in the creative process, if we're doing it right, it illuminates the path to future better failures. Maybe not solutions. So there's a few different lenses through which we, I think that's, there it goes. We're approaching failure. And one of them is as game players, right? Games are the medium of failure in a lot of ways. How we kind of figure out how they work by being bad at them. And then over time, we get better at them. We start figuring out what's going on. That, oh yeah, those totally looking things, maybe those we ought to avoid and those coin things. I guess those are okay. And I can't knock this stair-looking thing out of my way. So I guess I should try climbing over it. So games turn failure into fun in many cases. Behind that alchemy, that transformation is a bunch of failures as well. So the process of game design involves multiple prototypes, involves play tests. One of the most important things when you're making a game is to put it in front of people and see what they do because they're gonna do really weird stuff that you didn't want them to or you didn't anticipate. And as we craft failure, we're failing ourselves. Those are our hands pretending to design Pong, I believe. It's special version, though, that's like a battle Pong with different kinds of paddles. And then we also approach failure as educators. We're between us, we have 40-something years of teaching in art and design schools. This was from a workshop I ran for the Mozilla Foundation of making game design more punk rock. I can't remember exactly what these people are doing. It was about 10 years ago when I ran this. But as art school professors, our job in a lot of ways is helping people accept failure, integrate failure into what they're doing to sometimes be sad about it, but for it to be a generative part of the creative process. Yeah, just like that Mario that runs into the Koopa, is it a Koopa, right? Oh, Koomba, okay, thank you. Yeah, I get them mixed up. You just said that on the live internet calling. It's okay, oh God, I'm in trouble now. But just like that, students are constantly running into stuff that they've never encountered before and they're failing and so understanding and learning about the world of design through failure is really important. So how do we deal with this? All this failure that happens in the creative process. We iterate. Clearly, if you try something and you fail, you should probably try again because the next time you'll have that much more information that you can incorporate into your creative practice. This is a classic iterative game design cycle and as John and I said seven years ago, we were like, well we know game designers iterate and they iterate in this really specific way. This is what most game designers would say is the game design process, right? You begin with an idea, you conceptualize something, you prototype it, you make some paper paddles, you test it, you try it yourself, you put it in front of other people, other game designers, depending on what you're trying to get back and then you evaluate the kinds of failures that happened in the test because they will happen and you keep going around that cycle. So yeah, we've been thinking about failure for a long, long time. This is us doing a little fun failing on our own. We're doing a very controlled scientific experiment. They're off camera, there's some folks in lab coats measuring what we were up to. This is actually a physical prototype of that pawn game that we were working on. Here's the best part of it. We figured out how to play backwards. It's great, we can tell you afterwards how it all works. So ultimately, we know how we deal with failure as game designers pretty well. But how do others deal with failure? So we began to reach out. We talked to a winemaker, we talked to a comedian, we talked to a skateboarder, a chef, radio personalities, toy designers. And we found out that they all iterate. They didn't all know the word iterate, we found that to be a somewhat specialized language bonus. We talked to everybody about it, like Randall July, I can remember, is an example. I think when we first approached her, it was sort of like, yeah, I'm not really sure what you're talking about, but okay, I guess I'll speak with you. And then she was working on her app, Somebody, which is this really awesome social networking app. And working with software developer, she started hearing this term come up, and she's like, oh, okay, I know what that is now. And yeah, I guess I'd do that too. I never really thought of it as such. She probably thought of it more like doing successive drafts of a film script, or doing sketches toward one of her artworks or something, right? But she didn't always identify with that word. And there were a lot of folks we approached that just weren't comfortable with dealing with that F word at all, that just, that wasn't something they were at a place they felt like they could talk about. In some cases, people didn't feel like they actually did fail, which is, I'm not sure that's true, but. We had some colleagues who said, we're not interested in talking to you about this because we don't fail. I think we raised an eyebrow. Welcome to life at a university. At the end of the day, okay. Yeah. So hence the book. But the interesting thing is that there isn't one size fits all kind of iteration that everyone was practicing. We spoke to people, we tried to visualize their process. We began to ask, well, if there's one way of doing something, is there something on another side of the spectrum? And we came up with these continua, these sort of like sliding scales, right? And there were five different ones that we were able to encounter by talking to the different practitioners from material and reflective iteration, which we'll go into in a bit of detail, to four others that are a lot of different, very kind of different mindsets entering the creative process. So the most important thing is this is not a design book that says, do this and it'll work. Everybody needs to iterate differently. We should very positively and materially and thank all of our awesome grad students who worked on this project with us, who four of their names are on the cover, they're Steve, Eugene, Tuba, and Carla, but also Gabby and Angelica and Shwang Shwang. Yeah, we had a lot of great support from our students at Parsons. So let's start with our first two case studies in the book. Alison Tosier is the number one, probably the number one wine maker, she happens to be a woman too, in Napa Valley and... Matthew Maloney, who happens to be sitting right there in the crowd at this moment, who is a stop motion animator and then more generally an animator and a skull ring designer and many other things, but for the purposes of the book, we focused on his animation. So I met Alison here at the Media Lab. She's a director's fellow and she makes incredible wines. We had the opportunity to taste some and I'll never taste wine the same way again. Alison's materials are nature, right? So for Alison, failure can come in a wide variety of ways, it can rain too much, there can be too much fog, it can be too hot. You might, she likes to say that you can only pick the grapes once, you can't put them back on the vine. So for her failure is really intense and in fact she's iterating over the course of an entire year, entire season, where she kind of begins in the vineyard, they begin to prepare the vines and they do all kinds of different things and she has to make decisions along the way. When do they begin to train the vines? What kind of other plants might they grow around them like mustard and other things that create nutrients for the soil? What are they gonna plant where? When are they gonna harvest it? And we talked to Alison about 2011. There any, what is it, onophile wine people out there? Do you remember 2011 and Napa? Okay, well it was a terrible year because this fog descended on the grapes and it stayed there and the grapes couldn't mature, they were kind of like not doing that well and they started to go out and take leaf blowers and start blowing the moisture off the grapes because you can get a kind of fungus growing on the grapes and that's a really big problem. It's good for certain wines but not for the kind that Alison was making. So she had a really hard decision to make and she ended up picking some grapes before it was time and mixing it, blending it with other grapes that were faring better, that were on another side of the hill that didn't have as much fog on them the whole time. And she mixed it together and she ended up with this wine that was incredibly complex, kind of vegetal and ended up getting pretty good ratings. And since then it's completely changed her process of winemaking. She would have never kind of changed her process, she would have just been making the same wine year after year if it weren't for Mother Nature throwing this left curve at her. So then Matt who we refer to as practicing reflective iteration which I think a lot of people might look at what Matt does and say, I don't see the iteration in there. The thing we focused on in particular is Matt has this tradition every fall of watching, what are you at now, 100? Over 100 horror films leading up to Halloween. And he has a particular order in which he watches these things. So he's got to watch the entire Friday the 13th franchise in order. There's certain directors that need to be watched in this kind of chronological sequence in which they produce these films. And this goes back to his childhood of not really being allowed to go in Blockbuster and not really be able to watch a lot of these movies. And so he started telling himself stories about what was on the box art if anybody could remember going into Blockbuster and looking at the VHS package, right? And at that point I think he started to realize that storytelling was something. He wanted to do with his life. And so he has this annual tradition he's been doing for probably decades at this point of going back and watching these films and it helps him get back in touch with why he went into this creative practice in the first place, but also helping him see how he's grown as a filmmaker. And he takes sort of similar methods when he's working on a project and he gets to a point that he feels like maybe he's losing a little bit of distance from it. He'll watch a Kurosawa movie. What was it? Seven Seven Ride, right? As a way to remind him of this one moment back in grad school when he was watching the film with a friend and he got started getting too wrapped up in it as just a viewer and his friend was like, no man, come back. Let's watch this as filmmakers. And so it was a way for Matt to kind of get his heads back into the right place to do the kind of work he's doing. And so there's not a lot of active doing, like with Allison, she's out there with grapes and tasting things and with Matt, it's all going on up here in this way of reflecting. He's almost got this repertoire of films that he knows what's gonna help him get to a certain kind of headspace that's gonna help him advance his practice to advance whatever the project is he's working on. So together these are kind of two poles, two of the five kind of continue that we talk about in the book of going from this very visceral nature's happening people, we gotta respond to what if you're driven just by your own creative passions? How can iteration play into that? How can you create tools for yourself that are gonna help you deal with and mitigate failure as you move through the process? We don't need all these slides to build. There we go. So again, here we are back at these five continue for the sake of time, we'll just talk about that one for today. So ultimately the main conclusion of the book is that failure is actually kind of a fuel for iterative creative practice. When you stop failing, there's no reason to keep going, right? So when we think of failure, we often think of it in negative terms, but if you're incorporating it into a creative practice, it really can fuel your further iterations. The end. Yeah. Thanks. Now we're hoping that Joey can ask us some really hard questions about failure and iteration. It's wet up here. You need some more water? I'm good. Thank you. So I think there's a bunch of different questions I wanna ask, I wanna start just with the title, right? So I kind of get iterate, and but I think one of the things to sort of channel Nicholas Negroponte, one of the founders of the media lab, he said he often talked about how incrementalism was our enemy and that if you could measure it, it wasn't interesting, and I think it's kind of the DNA of the lab to sort of look for nonlinear leaps. And so, and you talk about a little bit in the book, but like how do you not be incremental when you're iterating, because it's sort of... Well, that's, I mean, that's the interesting thing. I mean, taking Allison's winemaking process as an example, she was incrementally improving her ability to make wine, but it wasn't until stuff started, like problematic stuff started happening that she was able to completely shift her approach. Her, like there was before 2011 and after 2011, and for her, before 2011 was making a certain kind of wine that people expected from this region, big, fruity, jammy, Napa Reds. After 2011, she realized that she could do something different, and so she shifted her goal. The first goal before 2011 was make these wines that are indicative of the area. The second was make something beautiful, and so that was a much bigger, more open-ended, less incremental goal, and ultimately, she ended up completely shifting the way that she makes wine. Yeah, I think the stakes are also too high to be incremental about it for her, right? Like there's hundreds of thousands of dollars on the line every year, or millions of dollars on the line every year, and so she can't just incrementally tinker. She's got to just do it, she's got to make the wine. They can't have a year that there isn't product. But I think the incremental thing, hearing you say that, it makes me think about the game design approach to iteration, which is extremely incremental in nature, right? It's always like, only make one big change at a time, see what happens, and it as a way to sort of hopefully isolate the effects of the change you've made, and it can lead to a really cautious approach, but I think as part of what led us to do this project was we didn't feel 100% comfortable with the Kool-Aid we were being sold by the Kool-Aid game designers approach iteration. Yeah, we were like, there's got to be other ways, right? Yeah, I mean, I think Miranda July is a great example in the book we talk about I think she was working on her second film, The Future, and she was getting really close to being late on turning in a script for it, and she just hit a wall, couldn't figure out what to do with it, and she's like, oh, I've got to make a film of some sort, so she's like, all right, well, I guess I'm gonna do a documentary, and she'd gotten interested in the penny savers. She's like, well, let me go interview some people. I'm gonna go buy this hairdryer, and I'm gonna take a film crew with me, and we're gonna talk. We should explain penny savers, this thing you get in the grocery store that used to, it's like Craig's list on paper. Yeah, we got back in the olden days. And so she went off into this project where she felt like she just failed on this one film, so she's gonna switch tax and do this other film, and in the process she ends up meeting some people that gave her the inspiration to go back to that film. Some of those folks actually ended up in the film. She also managed to produce a book out of this project about these interviews, right? And so she is not at all incremental about what she's doing, right? And it was looking for that sort of vibrancy and different ways of thinking about how creative practice can deal with failure. I mean, what I like about Miranda's process is that she's iterating through ideas with different projects. So she's taking one idea and constantly dipping it into other forms, whether it's writing or art. Or making a pop-up charity shop in a department store. It does seem that different products are different in terms of how they perform under incrementalism. Like games are traditionally better as sequels because you're building on top of the old game whereas films, most people would rather not see the sequels as much as the original. And I think there's something interesting about that. And actually Baratunde, who's in your book, who's also a director's fellow, he had this really wonderful, he gave a talk here and he said the thing about the problem with jokes is unlike music, is you don't wanna hear the same joke over and over again. Whereas songs, you wanna hear that same song over and over again. So I think it's different kind of what the role of the thing you're making is in sort of the narratives of society too, right? Yeah, are you making something iterative, baby? Are you iterating on iterative stuff? And I think that kind of gets to your story about Allison is do you want like the old Holiday Inn, the best surprise is no surprise, Merlot? Or do you want the, oh my God, this year's Colgan is crazy. And I think that also requires a normative shift in what you're expecting from the thing, right? Cause I think there's some TV shows, there's a TV show in Japan where the retired Shogun will walk around and something horrible would happen and then he'd pull out his medicine box that had the crest and then everybody would fall to their knees because Japan loves authority and they would say, oh, he's here and then he would dish out the justice. You should go to, they'll have just been in the middle of a fight or something and you should go to jail and you should marry her because they're in love and the thing was that my friend was a director of this and they would show it every day and it was the only TV drama that rated higher than baseball. But they changed the time that they pulled, he pulled this out, it happened like it's X minutes into the show and the ratings went down because everybody was waiting for the moment where this thing would come down and then justice would be served. And it's also interesting because I have a two-year-old who watches Coco over and over and over again and so she's so excited when the cat is about to come out. And it's, again, it's like she's watching movies as if they're, well, it's like Rock or Horror pictures sort of, right? Whereas most people don't watch the same, well, some people do, don't watch. So I think it's kind of interesting to think about sort of the normative context of some of these things. But, sorry, that was just not really a question but a statement, but I guess, but in your title, so you keep talking about failure but I didn't hear any failure in the two stories you said. So what was the failure in Matthew's case of watching films over and over again or Allison saving the day and not failing? Yeah. I feel slightly, literally and metaphorically under a spotlight with Matt sitting right in front of me as I talk about Matt. But often he's trying to deal with really complicated topics like he spent a ton of time in Hong Kong and so wanted to have this film kind of meditation on the way the city was changing and he wanted to experiment with techniques and do a live action stop motion film which doesn't actually make any sense when you think about it, but if you watch the film it's called Lune, you'll understand it. And so he knew he was setting up to do something close to impossible. He knew failure was right around the corner and indeed in some ways I think he thinks of the film as a failure. I think you think of most of your films as a failure because you're trying to do stuff you don't actually know how to do, right? And so I think for me as somebody who's been making stuff for 35 years at this point I view almost everything I've made in some way as broken but I've had to learn how to deal with that of just being able to sort of move on brush myself off and keep going, right? But I think your criticism is one we heard a lot as we were working on the project is but you're not surfacing failure enough in the way you talk about it. And I think it's partially because we tried to focus on people who had processes that internalized failure or built off of failure. Yeah, I mean we're still waiting for Allison to fail. But I mean for her nature failed, right? The materials started to fall apart and at a certain point they couldn't control things. So she had two choices. The first is to try to make the wine she always makes and a lot of winemakers in the region actually called in people from Bordeaux where the weather conditions were more similar to the problems they were having in 2011 and they did okay. For Allison, she actually took a second choice which was let's express what happened this year through a glass of wine. And let's not be afraid of the stuff that's kind of weird, kind of different flavor than ones used to. And for Allison it turned out okay because Allison is probably, she's one of the less, she's not very scientific about her winemaking although she is in many ways but she's not analyzing the chemical compounds of something, she's out in the vineyard tasting stuff. And so she's figuring out like let's pick this and I think we need to mix it with this because these two flavors will go together. So she's kind of mitigating failure all the time through her ability to taste. So it's maybe then a lack of a fear of failure than it is just having to fail each time to learn anything. And you showed the picture of this sort of big failure and I think about technical debt. I mean things like Vista where it just becomes so huge and they fail because people are afraid to refactor, afraid to actually have small failures. And so I mean is this, I guess it's, because in your book you talk about sort of the different sizes and the different types of failure and I guess it's sort of, I mean like we can maybe talk later about the faculty who said they never failed, that's kind of interesting. I don't know if they don't fail because they don't take risks or they don't, I mean like I guess the sort of meta question is I feel like if you have a process that mitigates against any failure you're sort of not taking risk. If you don't take risk you're likely to create things like technical debt or design debts where you're just piling on and not really moving anything but I mean is that appropriate? And then I guess the question is, what's the risk and how do you nudge a system to, because that's the problem, right? If you wait too long the failure will be larger. It's kind of like earthquakes that if you have little tremors you don't have the big ones. Is that a metaphor correct? And then how do you induce that? I think it's so much about the stakes of what you're making. In this case all the people we talked to weren't making things that were life and death. They weren't designing bridges or building software that would change the lives of lots of people. But part of it is also when you fail and we had talked earlier about moving fast and breaking things as a mantra for the risk taking nature of technology. That when technical failure happens along those lines then even if you took risks early on and quote unquote broke things there was a lack of responsibility, right? Like part of, and in the book we go into detail about what you do after something fails and one part of it is attributing responsibility. Saying, oh, that was me, we failed, right? And sort of owning that failure and moving on. Yeah, I think that's a big part in the chapter about failure in the book. We talk about that, that not everybody's in a place they can embrace it and accept it, but that the whole fail fast mantra I think is something we're pushing against in the book. That has a lot of the same baggage design thinking does where people think it's a recipe that is gonna always create success. And it isn't just cause you bulldoze over something and leave a bunch of rubble behind doesn't mean you're gonna actually have reflected on what happened and learn from it and maybe become more humble because of it. Because it kind of has a tinge of irresponsibility where your point about attributing it, I mean I think your point about attributing the cause is assuming they're gonna attribute it to themselves, right, so I guess not, right? And I think that's a key learning thing. And I think the problem also I think in academia even is when the stakes, a lot of our students are in the process of working under a PC, so I would urge you to be thoughtful about the claims that you make. Because I think one of the key things about science, and I mean like in climate science, I remember in the old days when even more people were skeptical, it was because the scientists, well it's still a theory. I think science always makes falsifiable claims that basically say here's what I know, here's what I can claim, here's what I can't claim. And I'm open to being challenged because that's how the thing moves forward. And the minute you sort of make it absolute, it doesn't work, right? And I think you need to set up the incentives and I think one of the problems with it just move fast and break things and not be responsible. I mean I think they do have learning built into their model is it is a bit irresponsible and it starts to screw up the incentives I think. And so I mean you kind of interested a little bit as you think, looked at different organizations, that maybe even reflect on your own new school where your colleagues don't fail. You know what do you think we can do in reflecting also for me on the media lab to create an incentive structure where people fail and iterate in the way that you think is appropriate? I mean the one thing I think really works well with game design and prototyping and is the play test. That's when you bring in brains that aren't your own. And those brains don't work like your brain. In fact, you're all game designers. You made games in the playground when you were a kid, right? Or you changed the rules of something and you saw someone interpret those rules in a way that was different than you. And you realize that there are different people out there. And I think that's really important. I think it's bringing in for Alison, it's paying attention to nature. For a lot of people with developing technologies or things that people will use, it's like let's bring in some people and see what they end up doing, right? It's like how do you inject a certain level of chaos into your process so that you're not hermetically sealing it and not open to stuff that's coming from others? I mean game design is co-design and it's not, there's not like one game designer. Everybody, the play testers are your co-designers as well. I think in our classrooms, it's ultimately it's about trying to create places people feel safe to be wrong. The creativity, to be creative is to be vulnerable, right? Like you're opening yourself up to whatever the world's gonna hurl at you in response to what you create. And helping people figure out how to be okay with it I think is a huge part of what we do as educators, right? And we were talking about this before, the education systems many of our students come from from around the world have taught them the opposite. That failure is a bad thing. That everything has black and white, there's a black and white to everything when I think we're saying maybe there's a lot more interest in the murky gray in the middle. And so there's a certain amount of de-learning or unlearning that has to happen, I think to get comfortable with this sort of iteration as a mitigating factor for failure. That was one of the things I noticed when I first got to the Media Lab, one of the favorite phrases here is it turns out that, and especially if they're talking about their own work, it usually means they tried something and it blew up, but it was really interesting and this is what they learned, right? I think kind of celebrating the discussion of your failure as sort of a fun thing. And again, I think it is important that the cost, whether it's lives or money is lower. And I think that's kind of the tech IT thing, is that you kind of get away with it because the cost of failure is lower until it's not, right? And but that was I think the general theory behind it. And I think that the other question is for these systems that are really high stakes, a space mission. Does this still apply? I think it does, we actually reviewed a lot of the engineering literature around failure and it applies, but the way of dealing with failure is totally different. There's like 30 steps and indicators and things that you do to measure responses to failure. It's a kind of very different process than one that's more open-ended and creative. But ultimately, failure is the learning moment, you know? It's like when you're a kid and you drop something for the first time, you learn about gravity. My wife is an artist and she's this project where she's trying to build a temporary pedestrian bridge connecting Brooklyn to Governor's Island. And she knew nothing about making bridges, but she's good at finding the people who do. And in the process, she's learned a lot and through osmosis and sort of sitting to the side, I've learned a lot about how engineering fields deal with it, where there's all sorts of test tanks, right, like who knew there was in the New York area two of these super high-tech pools that help you evaluate what's gonna happen to a structure when you put it on water, right? And so that's the place that you go do the learning, you do the failing at scale in a way that the resources at risk are controlled, right? So that when you get to the scale model you've solved for all of the stuff you can possibly think of, right? But then that's where this super elaborate 30-step process comes into play that when something does go wrong, there's a very rigorous methodology for trying to avoid Atlanta's highway collapsing again. And part of that answer seems to be don't store flammable materials underneath the thing you don't want to burn. Seems pretty obvious part of the solution in retrospect. But the learnings, I think, are really important. And I think just one thing I was looking at is sort of a lot of these days is sort of old traditions. And I'm from Japan in Shinto, we have Shrine. Shinto's 3,000 years old, we have a Shrine. It's a couple thousand years old and 1,300 years ago, they started rebuilding it every 20 years so they can continue to teach the next craftsman and then they can plant the force. But what's interesting about the Shinto is they actually put these stones across Japan at where they believed was an important high watermark. And so do not build beneath the slide. Wow. And we did. And that's where the last tsunami came. Oh my God. And it's interesting, putting stones with signs seems like a pretty important way that you could document it. And maybe we would learn, but then you just sort of forget. And so it's interesting how when you go across long periods of time, and maybe that's why Matthew watches the movies again to sort of invoke those learnings. Because I think that's the other thing is when you have a very complex system. And so engineers do it through sort of protocols and processes, but as I think about the lab, I've been here for eight years and I sometimes say remember when and then I realize that the students come through every two years, right? So we don't have a lot of documentation at the lab and even if we did, no one would read it. So it's, and I guess engineers, you can create a certification that you have to read this in order for you to be in charge of a space mission, but how do you think the learning, and there was a section in your book about sort of mastery as well, either as individuals or as organizations, how does that learning get recorded and transmitted? And then talk a little bit of the mastery part too, because I thought that section was interesting. Sure. Gosh. Yeah, I mean, I think the learning, we're looking at our specific case studies where people who have a life cycle, not necessarily institutions. So I think maybe we have to have a second book about that, but ultimately the learning gets kind of translated into the next thing that you're doing. And the way that people learn, I mean for Allison, she keeps a calendar and she talks about like, she looks at like five years ago, oh, I remember there was Fogg five years ago, let me go look at the calendar and see what I wrote down there. So some people write it down and actually use it again. I think other practitioners we talked to. Like Wiley, that's a good example. He talks about Wiley Dufresne, the kind of modernist cuisine chef based in New York. And he talks about cooking being a team sport, right? Very much they tend to work on this very old workshop model of you come in as somebody who peels potatoes and over time you move your way up and the only people who aren't a teacher are the ones at the bottom. Everybody else is teaching whoever's below them as you keep going. And you leave one restaurant, you go to another and you can't help but take the learnings and the practices with you to this new place. Everybody on his staff has a notebook that at the end of their time at the restaurant, Wiley or whoever will take it, they Xerox it and then they give it back to the person knowing that they're gonna be able to carry that on. And to me that's one of really interesting ways is that this interplay that's happening within that industry of how practices can move around, how one might learn to deal with failure. Wiley seems to be somebody that's comfortable with it, not everybody's going to be, but. Yeah, Baratunde was interesting too because Baratunde will be working on something like a script or a pilot. And ultimately what he told us is that he needs time away, he needs like extra time. He was like over a year late on his book. I think we all, we were more than a year late on this one. We can relate to that, yeah. But he said he needed the time away from it to return to it with like new eyes. So in some cases you don't wanna stay within that linear mode of learning. You need to return as a different person to see what's working and what's not. But that sharing thing is really important. One of my, the game industry likes these post-mortem talks, right? Where you come in and you talk about how are you major game. And I find the form full of potential but usually just really frustrating because all you hear is yeah, a couple things went wrong, humble brag, we have the best seller now. And you don't actually really get to hear because of the way the nature of that industry is so proprietary about everything that nobody ever actually gets to really talk about the juicy stuff. Well can I ask you a question about that? Because I talk to a lot of people who are involved in Silicon Valley and you definitely have a survivorship bias where you just talk to the winners and they often aren't reflective so they don't really know how they got there. They just think they're great when they were just lucky. And also like some motion picture companies like I think Pixar, we had some movie producers here were just talking about how they kill everything very early in the process so they only make mostly hits, right? But what you wanna hear about are the failures, right? And you don't wanna hear about the sort of like you said the little incidental failure in the super hit. It's actually why did this movie completely flop? And in the game industry, I mean in the thing that you're talking about, do you actually make people come out with games that are complete wipeouts and talk or do you listen to the hit people? Mostly it's the hit people but there's a fellow Chris Hecker that runs a session at the game developers conference every year called Failure Workshop where he tries to get people who are comfortable to talk about how the game they spent a decade on never actually came to market or that the game they thought they were making turned into this other thing but it's really hard to find people willing to go up on a stage in front of 500 people being live streamed to the world and say yeah, that didn't quite work out. And just a side thing, I mean the drug companies don't publish their failure results. And that's kind of life or death. You only hear the stuff that's successful and it's very trial and error. And so I do think there are a lot of industries that suffer from this inability to share their failures and I think that's... Hopefully we can destigmatize failure. Yeah, I think it's trying. I think it's weird because there's a whole kind of subculture of people who kind of love talking about failure. They're almost like failure fetishes, yeah. And then there are others like your colleagues who say they don't fail. But yeah, I mean, where do you... Again, it was interesting because you didn't talk so much about failure in the examples but it's built into the learning process, right? But I guess I have another question because I think if you do an A-B test, the one that's less successful is a failure. So that's sort of a very low cost failing mode and getting back a little bit to iterate because you have the word design. I've met a lot of engineers who really think they can A-B test to a solution, right? And this gets to kind of like, I don't think Allison would have A-B tested her way to doing that, right? And so again, and sorry, I'm gonna pick a few things from it. And also you talk a little bit about creativity and how back in prehistory or sort of in the Middle Ages, that would have been a blasphemy, right? We were the illuminated manuscripts were just taking knowledge from the past to the future and individuals weren't considered to be creative, right? And so what happened and what do you think? What this whole, because I think because you come from a design school, you kind of assume that people have this notion of creativity and not only your thinking, but in an engineering school, it's not that obvious and so how do you talk about creativity? I mean, it seems obvious that if you're like an engineer and you're thinking statistically and thinking about possibility spaces to take two points in a possibility space and then make your decisions based on which one works and which one doesn't means you're missing out on all of these other possibilities. So for me, A-B testing gives me the shivers because... I mean, Google for the longest time argued that's how they did everything, right? Yeah, right, right. Well, maybe you can choose a button color that way but I'm not sure you can make something new. That's what A-B testing is, again, to take a binary view of the world. Yeah. Yes or no. There can only be one yes in this, right? In that, that's not, I don't know, that's like so many potentially great things are just left to never be explored because of that. You know, in the game of Lost Words we're working on right now. We're part of the slow making movement, I think. This book took seven years. Lost Words, we're literally in our 10th year, I think, of working on it. It's taken many different forms and there's been a number of times where it's like, why did we decide to abandon that version again? That it seemed to be working but somehow we just... But you still A-B test your ideas, right? Not really. We just play test them. We play test our ideas but it's not like an, let's see if this book cover should be pink. Let's see if it should be blue. We never do that. But we'll put our proposition forward of what we think it should be. We gave a presentation recently at the Game Developers Conference called 100 Versions of Lost Words and there literally are 100 versions of this game that we're working on which is why it's taken us 10 years. And guess what? 99 weren't fun. We finally figured out a version that's somewhat fun but for game designers it's really challenging, right? Because it's like that there is kind of a, you can't just try two things and choose the better one. You have to actually find this moment that you're looking for, that you're seeking. But the whole kind of, how do you teach creativity thing? Like in the program we teach an MFA design to technology. There's some people with an engineering background. There's a lot of people with like a communication design, industrial design, product design backgrounds but there are people with like poetry backgrounds or American studies backgrounds, right? And one of the things we learned a long time ago is nobody is gonna come in with a sort of full arsenal of tools that are gonna allow them to work in their medium to know the kind of work they wanna do to understand and create a process. And so that's a large part of what our curriculum ends up being about is how do you turn into a reflective practitioner to use Donald Jones term? And a lot of it boils down to time on task in some way, right? But not in that Gladwell 10,000 hour sort of way, right? We talk about that in the book also that Gladwell's glossing over, he claims he's not but I think the way that notion of the 10,000 hour rule is taken hold in the popular imagination is you just have to keep doing, you just have to grind and you'll wake up 10,000 hours of grinding later and poof your award-winning chef or a skateboarder in the X Games or showing films in international film festivals and it's leaving out the most important part which is the reflective piece and the word practice and it's most basic definition, practicing and being able to practice in a way that you're learning from it and not just repeating the same thing over and over and over again, right? I find one of the things my students don't spend enough time on is the after failure point where they're, usually they fail and they just try something else, right? But they don't spend enough time meditating on that failure and really trying to understand what happened there and I think that that's something that we found. Most of the people we talk to, they may not even called it failure, it was like that failure encompassed their entire process in a way because they were never satisfied with what they ended up having. Well, it is they hit a failure and thought it meant they had abandoned ship, right? Okay, well this whole thing just must be thrown away and I must change tact here and do something else as opposed to spending the time to figure out what worked, what didn't, within the whole, which is really, really, really hard to be able to sort out. I realize we're starting to get short on time. Do, are there any questions from here or on the internet? There's a question up here. You just talk into that. Talk to the box. I have a minute question for you guys, which is, I guess my question has to do with the meaning of the word failure and how highly situated it is. So, you know, for instance, if you're trying to make a commercial game that doesn't sell well, that's a failure, but if you're making an art game and you're not even trying to sell it, the number of people who quote unquote buy it is not that relevant. So I'm just wondering in terms of a larger kind of contextualization, where do you, how do you define failure and for whom, by whose metric? That's an awesome question. I'll give our game design book a quick plug. One of the things we talk about there in that book is design values of basically knowing what it is you value about that project, what your goals are for that project and always having these as kind of guide posts for what you're looking for to come out of it, right? And you can, as a creative, you can set your own sort of goals or measurements for what that failure is going to be. I think most people, what's dollars or recognition or, you know, like the obvious sort of societal metrics, but I think as creative practitioners, that's really the most important part is establishing for yourself what equals success, what equals failure. I might make a meta observation. It's interesting, because you invoke the term a creative process, which I don't think I've ever heard at the Media Lab. And I think at the Media Lab, we mostly talk about serendipity. I think, like, Ed Boyden talks about this all the time, but he thinks that the majority of interesting things he finds while looking for something else. So he attributes to luck that which designers attribute to creativity, I think. Yeah, yeah. But is it similar? Yeah, well, I think if failure fuels it, right? Failure are those moments where weird stuff happens, right? So it is serendipitous, in a way. And maybe that's how we feel. Failures are really these moments when the world pushes against us in a way we didn't expect and we are forced to change the way we think about it. So that might also be a big part of it. It's kind of a non-linearity, whether it's walking away and coming back or doing whatever it is you do to your brain to be creative. Absolutely. I have a quick question. Probably a long answer, but not for here. It just hit me that we're talking, you mentioned climate and you're thinking about failure. So I'm wondering, maybe we need to put those two together and recognize that what's happening in the world right now with our climate is our failure. And so how do we become creative in thinking that way recognizing it and trying to find whether it's game design or whatever has to go into it to start pulling out of it and making it something that we can solve or... Yeah, absolutely. I mean, I think the big problem that we've had with climate justice and activism has been the attribution of failure. Like why is this happening, right? There's been a lot of arguments over whether or not this is because of humans or because of other things. And I think a lot of it is... The problem is that until you take responsibility for a failure, you can't really move forward to fixing it. So I would say that's the first step. So let's identify why this failure is happening so that we can change it. I think it makes me immediately think of Danela Meadows who's been an inspiration for us and much of our work in her kind of systems dynamics approach to dealing with climate change, right? Like she was one of the early folks who saw creativity as a tool to help us recognize the failures and the situation we found ourselves. I mean, that was what, almost 40 years ago, that Club of Rome stuff was going on. Yeah, I mean, her essay, Leverage Points, Places to Intervene in a System, I give to almost all of my classes because it really talks about how you can change a big problem, a big systemic complex problem. You don't go in by just sort of like changing a few variables. That's like a very small incremental change. You have to actually change paradigms. You have to change the way people talk about things and understand them first if you're gonna make a big difference. And in a way, I mean, what's interesting is things that you have control of to a certain extent games, for example, where you're the god-like architect versus a film which you have a little less control of how it turns out to ecosystems. And this is Don Elamedo. She says you have to change the paradigm, the paradigm of capital returns and extraction. And you change paradigm, I think, by changing culture. And you change culture, I think, through artistic and societal cultural intervention, which is much harder to design. And you can't really A-B test your way to them. And so I think that that's the other thing that's kind of interesting is, I mean, I think you have some people. Like wine is also quite complex. But I think our ecosystem is probably the most complex thing that we have to deal with. And I think Don Elamedo, to me, also is the perfect way to think about it. But it has, I think, a slightly different multi-party distributed design process that maybe should be your next book. Yeah. I'm working on it. I also want to ask him a question. But yeah. Hi. I'm kind of interested as to what you both think about. And this kind of came up earlier when Joe was saying that all of your failures are kind of talking about successes is what you think about failure for failure's sake. And so instead of failure being something that you use in a way that's ultimately kind of productive, failure is being a way of protesting or changing how people think about success. Because I think that all of the examples you've given are not really questioning kind of the fundamental idea of, like, oh, you should be a productive person and do all of these things. And when other people write about failure, there's a lot of discussion about the art of failing and kind of the intentional failing and things like punk and all the rest of it. And hold that thought because I have to go give another talk. But I'm going to leave you with the questions. OK. Thank you, Joe. Thank you, Joe. So there's actually an example in the book. I can't remember his first name. But his last name is Dedaminici, who's a performance artist. And he had found this weird trick that he was able to dial a phone with a piano. If you played the right notes, then the back when phones ran by tones, he could do that. So he had a performance. He went on stage. He's like, this is going to be amazing. I'm going to call somebody by playing some notes on a piano. And something about the acoustics in the room or the tune of that piano or something didn't let it work. But he kept trying and trying and trying. And he was like, man, the crowd is going to throw eggs and rotten tomatoes at me. But instead, they were with him. And they were in it. And they were fascinated to watch this guy struggle and fail to do this thing. He proudly told him he was going to do. And it made him realize that, oh, wait. There's something here to mind about this. That this can be a kind of work. And so a portion of his practice became about making failure opportunities for himself, which isn't speaking to the sort of protest stuff. But of somebody seeing failure as a way to get to places they wouldn't otherwise. Well, I mean, we do reference Jack Halberstam and the Queer Art of Failure, which is a big inspiration for both of us. And the idea that failure is a tactic. It is a way of being that we can embrace. And I think that's really key. And I also think that productivity is overrated. We like to make things that are counterproductive. They're called games. They're a waste of time. And that's part of what I'm doing for society, trying to make less productivity in the world. But I do think that we interviewed people who are good at what they do. And I think that's one of the challenges with the book itself. But we do talk about failure in its own incarnation as a special thing. Of course, I want to take advantage of Matt being here in the crowd. Matt's also an art school prof down at Svan College of Art and Design in Atlanta. And he deals with a lot of young, idealistic kids who probably think, I don't know, they're going to make the next Toy Story film. But they've literally never made an animation in their lives. And it's like, so how do you not crush their poor little souls when they think they're going to make their magnum opus and literally the first thing they do when they barely can even draw? Right. And they have to face the reality that they may never be satisfied with the film. Because there's so much growing personally that occurs when you're working on a project that takes that much time. It takes a year or more to create. Inevitably, you're going to finish it and look back and regret most of the decisions that were made. You're going to be twice the filmmaker at the end of that process. And if you were to re-approach that project, you would completely stand it on its head and rebuild it. How do you get them to separate out the amazing deficit of craft skill from the ambitions that they have? How do you keep them from giving up and going off to accounting school? So it kind of speaks to what your answer to that last question. So much of this, because we can misunderstand this idea to be, oh, just fail, grind at failure for 10,000 hours. And unless the, I think first there has to be an understanding that failure is a product of the creative process. So with that understanding, we can recognize it. I think oftentimes we may be failing at a micro level, not even be aware of it. But then on top of that, knowing how to harvest failure when it happens and make it part of the creative process itself, because it is a product. So you're either tossing all of that value away, or you have a system of understanding as to how to integrate it into the creation of the work. So when that failure does occur, when it's inevitably happening, you're right away thinking in assessment and then rebuilding that the framework of the question based on those failures. Cool. I think that it's a really great time to meditate at the end of your semesters. For those of you who are students here, on how you're going to harvest the failure that is your dissertation or the other things, because you know. My dissertation is a failure, for sure. Oh my god. Yeah, I'm so embarrassed. We are going to have to let the live stream go. So with your permission, I want to say thank you, John and Colleen, for all your time. With love with the book, may not be a failure, maybe. Or maybe it'd be a failure. Let's hope it's not a failure. If it's not a best seller, that's a failure. But there will still be available for questions. And we are just going to let the technology go and then you guys can chat with them. Thank you for coming. Thank you. Thanks, y'all.