 Welcome everyone, I'm Dina Zanfalia, a PhD candidate in the Design and Computation group here at MIT. This is the first lecture in the Computation Lecture Series, organized by my colleague, Shara Zaman, and MIT professor Teri Knight. Unfortunately, Shara, or fortunately for him, Shara is in Istanbul in Turkey, exhibiting his work, which is actually related to the lecture series theme and today's lecture actually. The lecture series theme is Wild Words and Tamed Minds. Proliferation of digital and immersive media in creative practices, increasingly challenges, established boundaries between the real world and the digital world. What was once imagined, created, and consumed in digital media is now leaking into lived experiences, calling for perceptual and material existence. The lecture series Wild Words and Tamed Minds explores this emerging type of creative reality in relation to the notions of computing, embodiment, and narrative. The series brings together designers, scientists, media, theorists, anthropologists, and philosophers to cultivate an interdisciplinary discussion on the theory of mind, body, and world. Today, I would like to welcome our first speaker in the lecture series, MIT professor, William Oricu. William is a professor of comparative media studies at MIT and also at Utrecht University in the Netherlands. He is a principal investigator of MIT's Open Documentary Lab. He explores the frontiers of interactive and participatory reality-based storytelling. His scholarly research considers the interplay of media technologies and cultural practices in relation to the reconstruction of representation, knowledge, and publics. A specialist in all media when they were new, that was my favorite part, he explores such things as early 19th century conjunctures between photography and telegraphy. The place of telephony in the development of television at the other end of the 19th century and the work of algorithms in our contemporary culture lives. William has held professional appointments in Sweden, Germany, Denmark, and China and received Guggenheim, Humpheldt, and Fulbright awards and most recently the Berlin Prize. Please join me in welcoming our first speaker, William Oricu. Thank you, William. So thanks very much, boy. That always sounds so weird to hear what one does. So first of all, thanks. I'm really honored to be here. This is a terrific initiative, although I have to say the tamed mind part worries me. I like to think I'm more deranged than tamed, but we'll have to find out what that means. I want to begin with a kind of double proviso maybe, or yeah, proviso is a good enough word. One, this is, it's not like this is earth shattering what I'm going to say, but it is the opportunity to give this talk helped me to try to think through some things I hadn't connected before and we'll see if it connects. I mean, I hope it does, but you'll be the judge. Second of all, they really pressed me for a picture, an image when doing this, and I just spotted this penal treadmill. These were used a lot, starting maybe in the 1820s and they kind of peter out by the end of the 1890s. And it struck me as a particularly insidious device, I mean a really insidious device. And I was intrigued by it and I stupidly sent it in and then it was like, oh wow, how do I tell a story about that? Like what's the link? It's pretty clear that both of these, the panopticon that we know from Jeremy Bentham and that Foucault has written about so much, we know that this is about an optical regime, right? This is visual control, that the power in the center can locate and fix visually and through fixing visually control what's on the periphery. The penal treadmill is a very different device, equally controlling, but it's one where what's insidious about, what's cruel, what's particularly cruel about it, is that by separating these prisoners, no one can control the pace, you can't do a slow down. Like basically you have to walk on those steps or you'll fall. And you can't slow it, if you could see one another you could kind of do a slow down and life would be a little easier, but you can't. No one knows who's sort of driving the pace. And it's actually the generative nature of this, as opposed to the purely representational or visual control through mere vision, this is really a generative process. And I'm going to stretch as hard as I can by the end of this to sort of argue that this somehow resonates with the algorithmic regime that we're in today, but it might be too much of a stretch. I acknowledge that right up front. So if you want to kick this overstretched metaphor, you know, I'm pointing to it. This has a kick me sign on its back, but we'll see. And basically what I thought I would try to do is to talk about what I see anyway as a shift in the notion of embodiment that we're kind of undergoing right now. I thought I would stick to the motion picture. I mean, I was asked to do it with film and VR and that's fine because it helped me to think about film indeed in terms of embodiment. Embodiment is a rich term. It's a term that's used in a lot of different fields means a lot of different things emerges historically 13th century 14th century. That's when the term pops up. The M in embodiment is an action prefix. It's about making something happen. The M in the body, we know is the body, the trunk or the chest already. It means that in the 14th century. And that combination is there to talk about materializations or feelings of materialization. And obviously I'll be talking about the feeling the perception side since these none of these technologies truly embody, but okay. So I want to just start with kind of the contrast I'm going to try to set up is between a notion of reaction on the one side and generativeness generation on the other side. Reaction in the sense of things your body does to media. Ways in which your body reacts that enable media to occur. And the generative will actually be ways in which media respond to the body. So that's quite a shift. In film media are there and our body enables the tricks and the conceits of the film media many way to work. But in some new applications that are emerging, including in the VR space, it's actually the media that is responding to the body. Not so much the body that's responding to the medium. And that's an intriguing shift and I see that as part of a bigger cultural dynamic. So let's just go through, it's pretty much like what I described in the little blurb if you read it. The first notion that I'd like to talk about has to do with duration. I want to talk about duration, I want to talk about space, and I want to talk about positioning, replacement, or setting. So for example, if we think of the Thalmatrope, there's a whole history of these 19th century, even 18th century technologies. And if any of you use this, you basically pull the string and it causes this disc to flip. And by flipping it, you see the bird in the cage. And we know that this was, you know, for years mistakenly imagined as persistence of vision, but that's not what it is. It's the five phenomenon. So, you know, people like Jonathan Crary have written about this. I mean, a canal is in a wonderful book, 10th of a second. Even Renee Lorraine Daston and Peter Gallison, and that if you don't know their book Objectivity, you should. It's a really terrific book. So what are the limits of perception? How does perception, how do we see things that really aren't there? Like the bird in the cage, or if you look at a phenocystoscope or a zoetrope, why do we see motion when there is no motion? And it's that perceptual, so I would argue that that's embodiment, that our body, by taking these discreet images and playing with the perceptual mechanism, by exploiting its deficiencies, its time lags, actually makes the trick work. The trick is not in the medium, the trick is in us. And it requires embodiment for this to happen, otherwise no trick. So of course, this is the foundation of the film medium. And whether we think about this in terms of just the perception of motion that occurs in those devices I just showed, or that occurs when you see one frame following another frame following another frame in a film, or whether it's the perception of causalities that occurs in the editing process. So the classic Eisenstein, Sergei Eisenstein-Potjemkin use where you see a soldier with a saber coming down, and you see the face of a woman, and what happens in our mind is that we've seen the soldier strike the woman, but of course we have it, or Hitchcock I guess is another, the shower scene in Psycho, where you might think it's a very violent scene, but there's actually no violence. It's a knife and a body, a knife and a body, and a set of reactions, but no knife penetrating the body. But the cuts themselves feel it. That is embodiment, right? That is a form of embodiment. It's a form of embodiment that's duration-based. It's based on a temporal sequence. It's based on, again, the perceptual proclivities that we have. And it's used in obviously tons of ways. The film medium is built upon it, but it's built at the end of the day upon a reaction on our part. This is a great film, by the way, if you haven't seen it, you'll understand more about it. This is a really phenomenal film by Peter Kubelka, and we're going to come back to Kubelka in a bit. So the second kind of embodiment that I'd like to talk about, reactive embodiment, has to do with space. And again, this is well trodden terrain in film studies, but again, maybe worth thinking about it this way. And it really goes back to the kind of development of three-point perspective, in a way. So we're back in the 15th century. So the book, it's interesting, the book, the printing press, Gutenberg's printing press, and Brunelleschi and all those guys, they're more or less within about 35, 40 years of one another. And both of these, and we'll come back to this at the end, but like the book, this notion of a world in which the relationship of the subject, the seer, and the object is mathematically pure. It's something you can really articulate. This is a human center, a subject-center worldview, three-point perspective. And it's interesting that both of these pop up at pretty much the same moment in the 15th century, because you might read them as technologies of self, right? The shift from a symbolic representational order in the Middle Ages, where the important people are big and the trivial people are small, to something that we would understand as more realistic, is based on an ordering of the world that privileges the human subject, not the cosmic order or not some other way of seeing. This is about the subject. Just as the book takes the voice of an author and amplifies it, stabilizes it to some extent, that's debatable, and amplifies it. So these are very interesting, it comes in an interesting moment, because this is the moment where the modern, in the sense of the big modern, the modern is being born. So Descartes with his analytic geometry is going to be someone who really articulates, this is the space he really provides theory for and grounding for. And of course he's the classic, the locus classicus of this object-subject split, right? So really one might argue, and I'll come back to this in the end with someone like Heidegger, Heidegger's notion of the Weltbild, the world picture, really is bound up in this moment, bound up in the relationship between the organizing subject, the seeing subject, and the world scene, as a very unique and distinct sort of position. And so in this case, the Dutch of course did a lot, among others, to sort of work out the math for this. And something like the camera obscura is a really pure expression of this. So what we today shorten and call the camera, the photo camera, I mean this is its principle. It's this seeing trick that reifies, that stabilizes three point perspective. You can goof around with the lens of course, but like in principle it comes right from this moment. This becomes the basis for a whole series of immersive technologies and for a feeling of embodiment. So if we go back to 17, I think it's 1787 that Barker does his patent for the panorama. This is a space that's built for the subject, right? The world is stable and no matter where you turn, you are the center of the universe. So it's very much about an enablement of a kind of embodiment of a spatial relationship. And yeah, this next one's not going to work. I had a tool, there's a great thing, Pano, I think it's called. There's a great little tool that allows you to turn panoramas into dynamic ones. And you can turn them and make VR things out of them. But I didn't pay my license fee and the demo stopped, so okay. This pops up in film theory in a really big way with, I don't know, the name may or may not say anything. So film has a, film's history is sort of plagued with reasons why it's, you know, not so, why it's trivial. Let's say it that way, why it's trivial. It's mechanically reproduced reality, it's trivial culture, blah, blah, blah. And in the 30s and 40s and 50s, there was a, well, so this gets redeemed in a number of ways. There are people who will argue that, well, it's mechanical, but it's actually not that mechanical. A German theorist like Rudolf Rheinheim will look at all the interventions that are necessary to create an image. Like, you have to make a decision. Do you take the lens cap off or not? Do you, what kind of lens do you use? Do you focus it or not? What kind of f-stops are you going to use? There's so, you know, where are you going to point it? What kind of stock are you going to use? Everything about it is a step, a conscious step that has to be taken. So to emulate the world, to mechanically, so-called mechanically reproduce the world, is in fact a fairly complex set of steps. And it's a conscious act. Someone like Rheinheim would argue, therefore distort the world and have fun. There are other theorists, Siegfried Krackauer, who will make a big argument about the inherent reality capturing capacity of film. And therefore, that's what we should do. Capture, let the world make its impressions. He's a believer in dexicality. Let the world be captured by film. But the guy who's really interesting to me is André Bazin, a French critic and maybe theorist. I think he was a critic primarily. Whose argument about film had to do with perception. It wasn't about what you see. It was about how you see. And his argument was that what film did well was emulate human vision by offering us deep visual fields. And if you let the shot run a long time, a long take, the viewer would have time to sort of enter the image mentally or perceptually. Enter an image and kind of figure out what was happening. You could focus here. This is from Citizen Kane. You could focus on little Charlie Kane outside the window, not knowing he's being auctioned off or sold off. He's being given to this kind of egregious lawyer in the middle of the frame. Or you could focus on the mother, the stern mother sort of watching her son being given away for a good reason. Or the sort of drunk father pushing back a little bit. Like where you choose to look is your business. Now from American cinema, what you get are relatively shallow shots that telegraph. It's much more like the durational stuff. The meanings are kind of telegraphed your way. European cinema often, and American, a few exceptions in America, Weiler, Wells, a few others, really would take a different approach where you could just kind of meditate on the image and let your eye wander. You could construct this scene in a lot of different ways, but you have to do the work by entering it. In a certain way, this requires embodiment. Much like the panorama, you're there, but it doesn't work unless you really understand that subject-object relationship. It's activated. It's tangible somehow. And this kind of film, this kind of conceit, this deep focus left with some duration is a great way to do it. Now, of course, some film forms, 3D, I don't know if you've seen Avatar, also really exploit this idea. Did any of you see Avatar? So what's interesting with Avatar in terms of 3D use is that where most 3D is stuff coming at you, Avatar kind of flipped it around and it was stuff that receded into the screen space. It was smart. It was a smart trick to do. But again, it just speaks more loudly to this point that embodiment in this case is a spatial trick. It's an optical trick, but it's one where we really feel as if we're kind of on the spot in a way. And actually, if you go to what's so intriguing with this panorama, if you... Barker's patent is a really wonderful thing. 18, what did I say? It was 1787. 1787, he does this patent and he says the patent is about, really it's about two things. Like you might think it's about the building and a circular canvas and all that, that's there, but it's like once. What he dwells on and repeats again and again are two things. One, the kind of perceptual claim of the panorama and he uses the phrase to feel as if really on the spot. And he says that like five or six times in the course of the patent. The goal of this thing is to feel really on the spot, to be being there. And the second thing that he spends a lot of time on in the patent is not the circular canvas. That's like one statement and it's done. It's the mid-ground. It's the space between the subject and the object. Like that's where the trick happens. It's that sort of scaled mid-ground that makes the illusion work. So all to say, these are very conscious endeavors to kind of construct that feeling of being somewhere, to construct that tangible sense of embodiment even though obviously it's an illusion. And there's André Bazin on the cover of Cayet de Cinema, a journal he edited for a time. And you don't have to read this. Just to say he's got a wonderful essay called The Myth of Total Cinema. And basically it's, I guess you could say it's like virtual reality or something. What he understands is the myth of total cinema is just reconstructing everything. So a lot of purists back in the day were upset about the coming of sound, hated the idea of color, like just, you know, the black and white silent film was the thing. And then really pushed back against that. And what he wanted was kind of this complete immersion into a perceptual reality that paralleled our own. It's a pretty interesting essay. Okay, third kind with cinema, again, that I'd like to talk about has to do with positioning, placement, setting. And in a way, literal embodiment, like what you do with the body. This is an incredibly interesting area. I have a book manuscript that I may never finish, but I hope I do just because it's so damn cool. Just to tell you, in the United States, New York, cinemas are, New York City, Manhattan, Manhattan alone has something like 600 Nickelodeons around 1906, 1907. And the mayor shuts them down by decree in 1907. Because there are a lot of reasons, politics being one of them, he's getting a lot of pressure about shutting down alcohol. That lobby is pretty strong, like it's very strong. So he needs to throw something to the clerics who feel threatened by backstory. A lot of immigration, kind of like a Trump moment back then. They were trying to figure out how to really suppress and control these unruly hordes. New York is something like 50 or 60% foreign born in this period. So they decide to stop entertainments. Anything that's legally defined as an entertainment. If you can shut that down on Sunday and everyone works a six day week, you're going to make people's lives miserable. That's the way they want to do it. So they shut down the opera. They close the museum. They do crazy stuff in the interest of suppressing these German singing, forines and whatever, all the immigrant theater. But by a loophole in the law, cinema is defined not as an entertainment, but as a machine. That's its tax status. And it thrives because it's the only thing you can do on Sunday. So the mayor really doesn't like this and decides to just, you know, and they were fire traps, it's true. These are pretty dodgy operations. So he shuts them all down. This is like 1906, 1907. And between that period and 1913, there is a pitched and very discreet and hidden debate about like how to legally articulate the cinema space in a way that's acceptable for society. And society in this case is the fire insurance industry. The fire insurance industry is basically threatens. They start with New York and they say to the alderman, they basically design a set of laws, a set of ordinances and say to the city, you must accept these, A, you must say that these are happening written in your name, that you wrote them, or we will double or triple your fire insurance rates city-wide. So the city folds. Now, what are those ordinances about? Those ordinances are about great things like, what's the width of a seat? Should there be a barrier between the seats or not? Because this appeared when there were often benches. Should there be a barrier? Well, in a dark space, popular heterosexual dark space, like back in the day, like yes, barriers and we still have them. How much light should there be in a dark room? Is there a legal minimum to the level of light? How much should the air change per minute? What's the volume of air change? What about fire exits? How many, where, illumination? This is all stuff. If you go to the cinema today, there's a residue of these laws that were first established in New York and then spread nationwide. The New York ordinance is what most other cities copied because of the fire insurance industry that mandated it. Key to this is a discourse about the body. I have to watch my time here. The little clock didn't start. When did I start? Like ten after? Okay. Key to this ordinance is like a lot of, an understanding of the body and embodiment, the kind of control of the body. So there's a lot of discourse of contagion. You would think contagion is disease and that's part of it. People spitting on the furnace and the tuberculosis-ridden spit bouncing back. There's that kind of concern. But there's a lot of concern of moral contagion that the underclasses and the middleclasses mingling in the dark could lead to, who knows what, sexual contagion. I mean, contagion is a very ripe word in this period. And a lot of the architectural regulation here is really about mitigating those kinds of contagion. So it's very much, it fits very easily into the Foucaultian notion of microtechnologies of control. I mean, it's very explicit and literal in this period. And, you know, there's a lot of behavioral, all the kind of instructions on how to behave and when to talk, regulation of time, speaking and you can't talk, you can't smoke. So a lot of embodiment of human form, the physical form is understood as something that needs to be regulated and contained. So this is a pretty big discourse. This pops up oddly enough and this is where Peter Kubelka comes back, the mad Austrian Peter Kubelka. This is a, the anthology film archives is still around, but not this cinema. So it's like a avant-garde screening space. And Kubelka basically borrows from a architectural motif that's used for 19th century prison chapels. So the poor prisoners have to go to chapel but they can't communicate, they can't see who's left, right, in front or behind them. They're trapped in a little box and only the preacher, it's bent them again in a way. Except Kubelka wants the pure cinema experience and again it's very much about the body and entrapment and of course there are more expressive manifestations of this address of the physical being, the placement of the body. This is Morton Highleg's censorama which kind of prospered in the 1960s. Seat would move, you get wind in your face with aroma. It's kind of a VR experience. VR is a very broad word. It's not just the goggles. And this was certainly, emphatically, a VR experience in the 60s. But very much about body, again, addressing the senses and the body speaking. And we see this today, and of course in today's VR forms of VR. Now this is kind of the installation version of this particular project. If you were to see it yourself, you'd probably use regular goggles. But this was the... And out in western Massachusetts, Doug Trumbull who's like a major special effects whiz in 2001. He's done mega films and he's got these sort of semi-secret studios out in western Mass. And his big thing right now is to find a way to do immersive three-dimensional experience, visual experiences without goggles in a collective open space. This theater, the MagiPod, will hold about 60 people. It's sort of a, you know, curved encompassing screen environment. It breaks down and pops up pretty easily so it travels around to trade shows or wherever. He's really looking at the capital right now to do it at scale. But again, it's all about when you look at what this thing is, it's all about where's the body, what are the sight lines. It's this very, very physical physical. You guys do this, the architects in the room do this for a living. But it's very much an acknowledgement of the physicality of the body, the limits of perception and whatever. And, you know, obviously VR environments like the cave do this, right? The cave which is the cave. So, all to say, these three modes are pretty familiar to, experientially anyway, familiar to us. But I do think they speak to very distinct strategies for embodiment that cinema has been doing for a while. And, that VR also does. VR is not the distinction I want to make here. There's a domain of virtual reality that uses all those old cinema tricks. So, I'm going to talk a little more about VR in a second, but I want to make clear that VR is not the difference between this kind of reactive and generative. VR exists in both domains. Okay, I'm going to slide over to the generative. The term here, I'm not happy with this term. I'm not sure it works generative. Ferrochi has a really interesting term. He uses this idea of the operational image where what he means is an image that actually does something, right? There's agency in the image. So, at least the stuff you work on would be primarily very much about this. Image recognition systems, tracking systems, those are spaces where the image itself is doing something. Operational image is actually a nice phrase and I didn't want to take it. And I also, it's got a valence. I don't exactly what we can talk about in Q it don't really like, but for now anyway, I'll use this generational. And there's a couple of different forms of this that are emergent. I mean, they're out there and like military always has this stuff. Generations before it hits the mainstream market. But eye tracking would be a really good example of how this stuff works. So, if you think of volviated rendering. So, right now in how many of you have used VR goggles? And how many have you done more than 360 experiences? Has anyone done like LiDAR or connect based stuff? Stuff where you can walk around around objects? Has it just been a world where you turn around? So, the stuff that's coming, it's all, it's in labs doing pretty heavy testing. There's even I mean, this company the aptly named Fove company. So, we're talking right now, most of us are pretty familiar with number one and number two 360. VR at last is a term that means a lot of things. And it can mean basically, basically Barker's 1787 panorama a fixed video environment and you were free to like look around. It's okay, that's or it can mean the stuff in the second thing that tends to be more real-time capture things like, so laser based LiDAR based or connect or photogrammetry where there's a kind of interaction you can move around objects. You can walk behind a chair. You can see what's on the other side of something. You can't do that in 360. But these are right now the two main kind of genres if you will. CGI of course, animated stuff. But the third one is where the action is. If you've ever wondered why Facebook spent two billion dollars on Oculus, it wasn't for the Oculus you can, you will currently buy. It's for the next generation Oculus that has eye trackers in it. Because, you know, Facebook's in the information business right now. Okay, we know a lot about your relations. We know a lot about the stuff that you're interested in or click through or don't click through. We know a lot from Google as well. But this will track what you look at. And it will, it tracks pupil dilation and reads rightly or wrongly. Reads that as a notion of interest, excitement and something. So it's not just the shoes, it's the red shoes. You know, that's the, so this is from a marketing campaign. And where we put our values, like gold, yes, it's valuable. And this is the new space for that. Whatever. The point is, though, that this is a space now where something's happening and it's not you that's controlling, I mean, you're looking at what you want to look at. But this is a system that's now reacting to you. Unlike the cinema system that we saw where you activate the connection between the bird and the cage. You make the deep space really what's just white and black stuff on a screen. You turn that into a space you can psychically enter. That's all your agency. Here, there's something happening on the side of the image. That's what I mean by generative. And the image, these tracking systems are responding to your activity. They're responding to what you look at and feeding you more or less. Now, this technology is being combined with a second kind of generative technology that has to do with where embodiment kind of plays out. And that's emotion tracking. So a system like Realize so that little camera on your computer when you open it up, there's that little camera. Realize their business model is to watch what you watch. To watch you as you watch ads and then judge what's your emotional response to the ads. Again, a huge source of income. Hard to believe, but it is. And it tracks, it's not just facial recognition, it's emotion recognition. Long history to this, we can go back to the 19th century and there are endless experimentations with trying to sort of figure out what are emotional registers as if these transcend time, space, and culture. They pop up in lots of different ways from a gestural system that DelSart works with to Sharko's work. Mental patients seem to be the most expressive and the most pure, you know, untempered by the moderation most of the rest of us. But these companies today, these companies like Realize have really made a, are making a sort of interesting business model based on just reading your face. So we have the eye trackers reading your pupil and your gaze and they combine with these things that are reading your face and you can write information, okay, marketing data, but also to anticipate where your interests might be. And that's the side of this I want to talk about. The emotion trackers take many forms, they can take the form of, as I said, facial recognition, but they can also take, they take a lot of other metrics, galvanic skin response, heart rate, skin temperature, right, that's really interesting data in terms of how you're responding to an experience. So if you think about where I just had a meeting I think it was last week with the it's being taped. So the president of a major, the Chinese the president of a major company their Chinese operation and they're in VR and he said to me, we were talking about great VR experiences and he said wow, I just saw this thing last week that really blew me away and it was it was a really compelling narrative and I was like, it was amazing and only after I did the experience that I realize that actually it was a very elaborate branching narrative. In other words, at any given moment in this thing you could have done this or done that, but there were no, from his point of view, no points of decision. No points where you go through the door or go out the window. No points where you talk to this person or don't talk. It was a seamless linear narrative and he was blown away by it and it was blown away by how seamless it was when he realized it's real structure. How did it work? With these technologies. A bracelet tracking his interest level, assuming his interest level based on heart rate and whether he was sweating or not eye trackers so that where he would look in the scene and if his pulse was going the system read that as you're interested in this, we just bleed in the next scene in that direction. So the narrative is being constructed is being generated based on these kind of metrics of your behavior. We aren't the one driving, you might argue we are, but it's actually an algorithmic intervention that's sorting out and assessing you're looking here, probably you want to know more about that so we'll give it to you. So this is what I mean by generative. This is a very different order of media experience and a very different kind of embodiment. It's the flip side of that cinematic experience. If we were to look ahead a little bit, there's some kind of interesting stuff happening and I'll just mention it because it's spatial enough that it might be interesting to this crowd. But it's really putting chips, we have a lot of chips these days and they're getting smaller and smaller. So this is a project that happened here with Sensible Cities where a chip is embedded in a piece of garbage plastic container and it essentially tells us a story of inefficiency, a story of ecological irresponsibility or responsibility. But if you think about the ability to ramp this up it can go to a pretty large level, so-called Internet of Things. One of its really interesting capacities is again to shift this is shifting that sort of narrative agency from the human to another space completely. So anyway, what I want to just talk about, you know, kind of wind down with is to step back from these two examples and talk about the bigger maybe the bigger implication. And I want to use Heidegger to do this as I mentioned earlier with his notion of the world picture. It's a lovely picture of him. I don't think I've seen a color picture of Heidegger and it's a nice one. Right, so his point here is the very fact that we can conceptualize our relation to the world as a picture is kind of the emblematic of the modern and it goes back to what I was saying at the outset this idea of a really clearly defined subject-object relationship. The thing that crystallizes in the 15th century that Descartes really provides the underpinnings for and that is with us today. This is about an order, the term that's used in the 15th and 16th century sounds like algorithmic but it's actually a different word. It's a related word, etymologically. It's algorithmic. There's algorithmic and there's algorithmic. Algorithmic is this domain of precision, of calculability. It's like what you want with your bank accounts. You want to know exactly what's in there. The algorithmic is all about a formula. It doesn't matter what value systems you feed into it. It's a finite recipe. It's a finite set of steps but the values are completely arbitrary. So the algorithmic algorithmic is really about precision, calculability, predictability. It's accurate, it's exact. It's the thing that the modern is built upon. The big modern. Algorithmic is contingent. It depends on what numbers you put in. It's very much about the formula, not about the content. So that notion of contingency and variability is really quite important in it. Just to put that in visual terms, if we think of Canaletto's Piazza San Marco, this is the modern, that old modern where there's a point of view and that's the world as it appears to an individual. More or less. But if we look at the algorithmic, this is from Photosynth. Photosynth, this is the old Photosynth, not the current one. Photosynth would take up to like 800 or a thousand photos of a space, look for points of algorithmic similarity and stitch them. So you know this from your Google Earth where a lot of images are kind of stitched together. Now in that case there's an efficiency. Like it's as few images as possible to construct a space. In that case, it'll just take random photos that are tagged to Piazza San Marco and find points of algorithmic similarity and just lock them together into a space that never looks the same to any time you go to it. You enter in a different space and it's very hard to recapture it. If you want to see a good example, there's one I think it's called for nostalgic purposes. I mentioned this. It's called The Moment and it's the swearing in of Obama. CNN or someone did this. And it's like 900 crowd sourced images that if you want to go off and look at the tuba player or if you're interested in Michelle Obama's gloves, you can just move all through this space. It's not a big image that you move around in. It's an image space. It's multiple dimensions. It's seething. You can't capture it twice the same way. So that's the algorithmic in the sense of being highly contingent as opposed to this very stable individual that's more this is kind of the construction of photosynth. You can see that there's a ton of tourist photos, different cameras, different lenses, different days point cloud is made images are assigned to them and we the user can kind of just kind of move around in this very again contingent space. So if I were to sort of tease out the differences between these two regimes this being the modern the modern from the 15th century until we still we still this makes sense to us very much makes sense to us we understand the idea of a fixed point of view we understand attribution authorship that makes sense to us who's responsible stability textual stability everything we do half of what we do at MIT is based on it if you're not if you're if everyone in the class is reading a different version of the text you have a hell of a hard time having a conversation this very clearly defined subject-object relationship that's our culture that's our philosophical order or epistemological order this makes good sense to us but where we're moving and what's creating a lot of headaches for some people is so I have a documentary lab that focuses on interactive and immersive documentary well there's a million point of views like there's no right way to see it there's no right path to take diffused our attribution these are often crowdsourced projects like what's the right who do you attribute this to when you have multiple authors collaborative text instability and of course the enablement here is algorithmic so this is a quite a different kettle of fish and we often look at this stuff as if we look at it through the lens of the modern of course we're modern subjects we look at these things as if but as soon as you start a discussion let's say with interactive documentaries what comes that well who's point of view and who's the author and what do we attribute and what's right you know the classic I mean we see this a little bit with Wikipedia and with Diderot we know who wrote the encyclopedia if he's wrong damn it we can say Mr. Diderot you screwed up you know this is not true we know who to grab we know what to correct but Wikipedia the bot wars that are going right now on with Wikipedia as things are it's morphing all the time and it's rich and it's fabulous and it's collaborative but it doesn't really hue to the to the characteristics of the good old stable modern I'm sure some of you in this room went to educational systems where you heard don't use Wikipedia don't cite it because it's too in fact it's pure back to page and you see exactly what's contentious it's a fabulous source but it doesn't hue to the characteristics that we're familiar with in the modern and so back to this back to this crazy image I don't know how this fits but I do think there's something interesting here about like these algorithmic interventions where we don't see them so in the good old days of the subject object life was relatively simple at a moment of algorithmic intervention it's a little harder to know your relationship to that object world because there's there's something between you and that there's something enabling like with Wikipedia there's things enabling this text to take form and it may not be there in five minutes or tomorrow you don't know it may not shift from region to region our car navigation systems I mean you name it there's this kind of contingency involved and but it's very generative and it draws on the power of many different people to make something happen our individual part is we don't see it nor do we see what's being contributed by others a little bit like this scenario and yet in the aggregate it's incredibly generative so we don't really have a good assessment system for that yet I don't know if that defends my use of this picture or not but I will show you this quote from back in the day from 26 and it's revealing it requires no previous instruction no task master is necessary to watch over the prisoners neither are materials or instruments put into their hands that are liable to waste or misapplication or subject to wear and tear and this is the point and it imposes an equality of labor upon every individual employed no one upon the wheel being able in the least degree to avoid his proportion and that's what I find so intriguing about some of these like these image tracking systems or these facial recognition systems data is being gathered like crazy or will be in his that's fitting into predictive systems that are generating texts for us that are producing things that in a way we don't really control I mean you can project it you can step out of it but that old simple mechanism of control that we had in the modern is starting to shift very very rapidly with these generative systems so that's a kind of embodiment that I'm really intrigued by and I think it is a very different kettle of fish than the Jeremy Bentham model which is really about reflection observation a regime of visual control yes but not about it's not generative in the way that the the penal wheel is and so with that I will shut up and thank you William any questions maybe we have 15 minutes 20 minutes for questions thanks for a great talk very inspiring so I have a question about you made a comparison between sort of the old you know stable regime and what's what's emerging and before that you talked about this one CEO of a company who was totally immersed in that driven you know this emotional embodiment and that gave him a completely new experience so I was wondering you know what kind of genres are emerging through that and because you made the connection between those branching narratives of course there was an author behind it or multiple authors behind it who made those branches happening so in your grid you said you know this is the new form is all collaborative so how does that play with the notion of the authorship in emerging new kinds of genres yeah good question and what I would say first is that the experience the guy had is probably like the best cinema or the best old school VR experience he's ever had in his life in other words he was not at all aware of the interventions that were being made it just made a super old school experience even though the mechanisms for it were super new school the collaboration comes from like even in a path narrative there's authorship certainly like the environment the crafted environment but we don't I mean maybe we need to expand our notion of authorship a little bit to think about environmental authorship rather than textual authorship but the text is actually the text he experienced is actually I would argue a collaborative one in the sense that he choices were made based on a aggregated data from other users that this looking here means something wasn't based on what he meant it's based on the previous experience and what the coders put into it and be his own like optical choices but he's in a conversation if you will with a with an environmental author an environmental creator so it's maybe two modalities of authorship the an actor of the of the experience the person who chooses how to connect the dots and the person or the people or whoever who made the dots but that algorithmic intervention is really a kind of data-based set of assumptions or extrapolations or or summaries that actually translate the kind of intermediate between what the user does and what the environment allows so it's just we don't have a good critical vocabulary for that yet I think I mean that's part of the thing we need to develop hard to talk about this stuff if it completely falls outside our academic traditions I mean maybe the mathematicians talk about it but questions and thanks so much for the lecture and I'm trying not to formulate the question but it's the first time I heard about the generative VR as a term so I don't know much about what they're thinking of is it a technology that's going to be used in like movies or like I'm not sure but it reminds me like this idea of the structure of having like more options generated by the user remind me of this discussion from like I was like open work I don't know if you're aware of it and on one hand I think it's like promise of more freedom or more subjectivity like this more like collaborative work based on this assumption this concept but on the other hand we can say that these choices will be like pretty much finite so like the system might choose between 10 options so it's not really it's like it's a pseudo like freedom or like is it really like empowering more of the subject or is it like more like imposing like a more structure in the end because somebody could say that any work of art or like any movie is open like because we don't see the same thing exactly so what's your good question so you're indeed right Umberto Echo is an important reference here and I would argue and have argued that actually this notion of an open text and actually a relatively I'm not a very much of a believer in stable textuality in general and the text I would look at would be things like the Bible the Bible like do you know how many just the Catholics and they're pretty orthodox just the Catholics have like 22 official versions of the Bible if you look over over the last century and now and across there's a lot of variance put that together with the construction of the Bible from its earliest days of multiple texts being in or out translation variations endless quibbling about there are there are like what the hell is the Bible it's like a really very very it's slow it's glacially slow the transformations and but you can get killed if you kind of embrace the wrong one so that's to me a very interesting notion of an inner a kind of textual interactivity where the interactor is a community or an institution over the long haul you could look at a shorter window at 19th century seriality where someone like Dickens Wilkie Collins would publish a chapter each month and then re-aggregated into a book but Wilkie Collins was notorious for kind of keeping his pulse on the public and if they didn't like something he would like change the route of the of the chapters and he would rewrite he would do rewrites before the final fully agree what this stuff is kind of radically different in this sense we're thinking of it through the lens of the branch narrative and that's indeed what I said but think of it from the lens of like YouTube 400 hours per minute a video or being uploaded to YouTube right now 65 point seven years worth of video or being uploaded per day we don't use it because it's like it's too much what if we had the taste predictors that we're using with Spotify with image recognition software with there's a lot of storytelling software narrative science is selling millions per year of newspaper stories anything with structured data like sports or finance they just feed the data in one side and outcome colorful stories on the other you cannot tell the difference narrative science is really going for more creative prose now so put those three together image recognition taste prediction and certification you can imagine being able to go into a data set like YouTube and come out with a different story for everyone based on whatever so I think yes looked at retrospectively relatively few choices it's what's the big deal looked at in terms of where the possibilities are and where we're starting to see the action there are some really bizarre and exciting possibilities out there thank you any more questions thank you for a great talk I really am interested in the eye tracking stuff I looked into this a few years ago but I haven't really followed it and I'm just wondering in line with your argument it's the idea that the image is this concept of generative embodiment also implies that the image is watching us that the image is the image is an eye is a sensor and gathering all kinds of data about us that then has all these IP issues but I'm just wondering what the visual culture of generative embodiment looks like because on the one hand it aspires to a kind of transparency and seamlessness like the way you were talking about the construction of a narrative in a VR environment on the one hand but then it's possible to generate a diagnostic view like to show the eye tracking report or the sentiment analysis on the face and I think it would be great if we could begin to publicize more of those diagnostic perspectives because it could generate more visual literacy around like how the function of the image is changing so the image as a report on you instead of something to look at I just think this is so productive and essentially you're making a kind of modernist with a big M, modernist argument like the way that the rupturing of seeing or the roughening of seeing so that we see more clearly in this case through data and I think part of the trick we're at this moment where we're kind of most of us are pretty culturally ingrained with the longest art historical tradition all of that's coming we're rooted in that older way of seeing and we're in a world of a lot of other possibilities but we are inclined to want just to make that old way of seeing even better and even more seamless and even more the adventure even more evocative and more custom tailored rather than one size fits all that's our inclination coming from the cultural space but what's possible on the other side of that divide what's possible with that to use modernism in a very different way but that modernist ethos where it's about where it's about seeing in a new way seeing astronomy or whatever a fairframing or whatever kind of term you want to use to describe that rupturing yeah there are great new tools there I mean the course I'm giving a VR course this semester and we call it hacking VR we don't want to replicate some immersive world and it's like this how can we find its vulnerabilities and a new set of expressive vocabulary like that's absolutely agree and data is key to that I think sort of feeding that stuff back is a great way to do it any more questions? thank you so much just let me thank you guys because it's like a it's Friday at five o'clock holy crap long week already it's nice weather outside I'm so impressed that you guys showed up my thanks to you so thank you very much