 Sit for a minute. When I was a little baby, and I did my first book, Muriel was at MIT Press, right just shortly after that. And she encouraged me. Moments of encouragement when you're really terrified and when you're truly stupid are very important. Muriel was with MIT Press for quite a while. She's been with the Media Lab for quite a while. She's always been two or three steps ahead of me with everything she did. And it's an honor and a pleasure to have Muriel join us today. Thank you, Muriel. I have a couple of, can you hear me? Yes. I have a couple of personal things and characters because of Ted's philosophy. First of all, I'm testing the boundaries to see how truly comfortable one can make oneself here. Secondly, I'd like to ask for some pillows next year. These are not anywhere near as cozy and comfy and raunchy as the one at Kobe. I have a confession, and I hope that everybody in the audience will forgive me. In those early days when we were both very young and terrified, I made the mistake. I remember how I did it, I remember when I did it, and I think I remember where I did it. But I'm damned if I know why I did it. I introduced Nikki to Ricky. So I should probably get off the stage right now, as a matter of fact. Muriel, if you think you were worried, may I please introduce my young colleagues? With a hand up, David Small. We're going to probably do the inverse or the obverse of what our young friend just did. And I'm going to try to go as fast with images as he did with text, with speech. I'm not sure I can handle it, but I'm very well known for showing too many things and talking too little and wandering around off the point. David was a student in the Visible Language Workshop. And he studied cognitive psychology as an undergraduate and got interested in image making at the LW. And resisted computers for a very long time up until, I think, something like three weeks before the deadline for the master's program, at which point he sat down and decided maybe he would see whether computation had any creative capabilities for him. And since then, he's been contributing some of the most brilliant and interesting examples among the many that we're going to show you in the next half hour, I hope. So rather than waste too much of my time, for those of you who don't know, the Visible Language Workshop is going on at least its 20th birthday, which is pretty shocking. And I feel very blessed, as I hope we all do, that instead of getting older, we're getting younger. And certainly the projects and the excitement and the enthusiasm of the environment and the media laboratory just continues to grow, at least for me. So that's a great blessing. Our goal has been for a very long time to try to examine in these so-called emerging technologies what the new form and content of design of communication might be. And to that end, for many years, we've been building sort of prototype and visualization tools that would allow us to do something that seemed relatively intuitive in order to say, what if we did this, then what would happen? But to really visualize the stuff in as tight and iterative a loop as we possibly could. So we're looking for new design principles. We're not at all sure what they are. Once you get into this new information environment, this proverbial information superhighway, and you have dynamic and interactive issues that we really never have dealt with before, one has to examine the conventional design wisdom. My own background is in graphic design, but one has to redefine graphic design. And one needs to be able to see whether, in fact, traditional design knowledge actually maps on to the kinds of things that we need to know and need to be able to do in the next 5, 10, 15 years when the technology enables us to do much, much more than we can today. So last year, we produced one of those dignominious design ideabytes. Yes, he crawled, and we unfortunately responded. But I'm going to show you just four little pieces from that to show you where we were roughly a year ago. And most of these are looking at dynamic and interactive problems that have to do with not being able to anticipate how information is going to come into the graphical environment. And so can you roll the takes? I can't have a pause on this, so I have to talk rather fast. So this is real-time weather data on a very large 6,000 by 2,000 very high resolution screen, which doesn't look it here. However, we're using geographical context, taking real-time news and locating it geographically and allowing it to build up so that you know where the most interesting news is occurring in a hurry. We're also clearly bringing in video and so forth. You could see, can you stop for one second? You can give me black if you would. Maybe not, never mind. OK, this is bad, because you noticed, I think, some of the pile-up of the text. One of the serious problems about not being able to anticipate, as you might in a book, exactly where things are going to be is that you keep these things pile-up. So there's a huge amount of very interesting design problems that say, how do we get away from this pile-up of text? And we'll talk about it a little bit more later. Now if you could start again. What's coming up next are two little examples of interaction with the machine where we're using like sound icons. Visual scanning and audible scanning are two very different things. One's very linear, one is sort of simultaneous. We lost the sound, terrific. This is by gesture where you take a number of intelligent objects and tell them where to go. And because they understand what kind of speed they have, what kind of weight, what kind of terrain, they follow a pathway based on their characteristics. So there's an intelligence, and we're looking very seriously at filtering information. This is layers of information where we're pushing through in the z-space. This is an overall high level way of managing 30 layers of graphical information by importance. So this is pushing back information that was unimportant for the task and bringing forward information. 3D has always interested us, and up until very recently we have not had the capacity. This particular example was done in the traditional way of building models and taking frame by frame and shooting them overnight and taking a lot of computational power and never being able to look in real time. This is an infinite zoom where we're looking into information and finding nodes of information. And it's been a kind of model for us. These past four things sort of sum up some of the ideas that we've been struggling with. And they've had a lot of problems. The advent of very fast real-time graphical machines such as the silicone graphics machine, which you've seen examples of already, allow us to begin to explore the boundaries between two-dimensional design information and three-dimensional design information, where one is valuable and where the other is valuable, how you move seamlessly between the two, and what kind of design problems evolve as a result of this capability. That last infinite zoom was done at great pains. You drew every single frame. Now you're able to do a kind of zoom in a very, very different kind of way. So David. We put up the screen here? Yeah, good. Oh, we still have the line. That's too bad. OK, we've tried to integrate some of the existing very, very new ideas that have evolved over maybe the past six months as a result of our having some incredibly good students as well as some incredibly good machines. We're looking at three ideas here. Wait, go just a little slow. And we're being concerned with spatial data, with symbolic data, and how they interact. And we're going to examine a few examples within that. But first, we're going to do a kind of quick, verbal run through, which is maybe like a dynamic overhead projection system. I'm not going to read these. And I think if you just go slow enough so that you can read them, this is just really an example of some of the interesting typographical problems that begin to evolve when you start to use three-dimensional and two-dimensional information simultaneously. You begin to be able to compress type. You begin to be able to zoom into it. And you develop some very interesting questions about what happens to things like type size, you want to say. What happens when you move around the type and it's backwards, and you have to begin to design. And make it intelligent enough so that it doesn't become illegible. How do you retain the integrity of the information? And at the same time, retain the context and the cues that allow you to traverse complex information. So here we go. I mean, this is the thing you should never do, right? Never do something that you've never done before. But if this isn't the place where we can do it, then I guess we have to go somewhere else. So why don't you just sort of zoom around and I'll. So this is, I think, an extremely interesting example of where you can read two levels at one time. You're taking advantage of the spatial relationships. And you should be able to traverse. You're in a sense in an architectural construct, but you don't have the constraints of having to believe of physical building. And so I think there's an enormous, it's a little bit like designing an exhibition, but without the constraints of the physical difficulties. You don't have to build it in the same sense. So you can both use the abstract conceptual issues as well as the physical cues that people are accustomed to. And I think it's just a wide open research field. We can rotate around this information. You can zoom into depth and multiple layers. And what it does for us finally, which, I mean, we're still finding out what it does. And in fact, just making this try to work in the past few days here, we learned an enormous amount. One of the things that you've been seeing a lot of is hyperlinking in some of these multimedia products and research things that people have been working on. They're all limited by sequence and by branching. The capability of this kind of environment releases you from the rigors and the sequential aspects of branching. And I think it offers an incredibly powerful way of going through this vast amount of information that the internet and other kinds of data sources are going to demand of us. Information is absolutely no use if you can't find your way through it. One of the questions that you have to ask yourself is whether who does the driving? Do you, do we, does the machine, where does the agent fit in this, what kind of filtering happens? And somewhere, I think, between some shared responsibility for where you're going in the machine is something that we need to look at very hard and build some graphical intelligence into the system. So are we broke? Should I keep going? And so if you've been reading while I'm talking, there are great students and great people and great groups. And you could, in fact, cruise this on your own. And we had hoped, had things worked out better, that you could have played with this. But I fear it's going to have to get broken down rather quickly after, right after the break. Anyway, there are lots of people to thank. So let's move into some of the, oh, yeah, sorry. Yes. Now I'm lost. Oh, God. There we are. Well, these are some of the issues. And I think I've already sort of talked about them. So why don't we just move into the example. What is your little, oh, 14 minutes of elapsed. OK, we have a timer in the upper right hand. But it doesn't do anything like slap my hand when it gets to be 20, 25 minutes. Because here's some of the work in progress. And we sort of broke it down into showing you some symbolic data and some spatial data. Again, there's a kind of compaction of information. Legibility is going to be a really serious consideration. But I think that the fact of being able to move around, change your viewpoints, change views of information gives you such a powerful way of browsing and navigating complex information that there's several PhDs here. So we're going to look at typographic space here, traffic control, and some network multi-views. And then we'll move over to the spatial data. If we have time, those last two projects are special. So interspatial data. So what we wanted to do was have another view of the media laboratory and say media lab meets headzilla. But we couldn't make it happen. So this is a very early example once we got our SGI's, beginning to see what happens to typography in a topographical space. And one of the interesting things that we couldn't show you, unless you could get a donation of about 500 special glasses at 1,000 bucks a piece, is what stereo can do to this and how it allows you to build up very complex information. So we're moving, again, combining some symbolic with some representation of information. And you see what an interesting problem. You know, if you're behind an ambulance, that works. But I'm not sure that this is particularly good. So we want to move over into the East Coast where things really are happening. And into Logan Airport, where maybe Steve and Jay Gould still is sitting, I hope not. And we're going to move, this should be a smoother transition, but it will be next time. And this is a small little graphical essay in three dimensions on air traffic control problems. This is your typical way of looking at it, the previous image. And this is adding 3D and some mixture of two-dimensional graphics, the abstract information, along with the representation. So you're seeing the height, and you're seeing the airplane. Are we through the air traffic controllers? So this is the Green Plains view. Right, so now you can see from many points of view, obviously. There's the air traffic control. These are the corridors through which the airplanes go, and you're tracking them. And of course, ideally, it should go without saying that ultimately these will be connected to changing data, and the graphics will change as the data is being fed. There's a green space principle of safety for pilots that those of you who fly probably know much better than I. And you note that we're charting the heights of these locations. And when they get too close to each other, the green space becomes red space naturally. And so this is a way of evaluating exactly and visualizing where these things are in relation to each other. And we've had a lot of air traffic controllers coming by to see how this fits their mindset. And they're very unique kinds of mental models that these people have developed over the years. And one of the most interesting things, I think, for the future, is how we can, in fact, visualize mental models that are appropriate and cognitive to particular cognitive styles. And yes, of course, we'll be talking to this machine. And yes, we'll be delegating, but it seems to me that the graphical representation of information, because it is, in many cases, so efficient and allows the cognitive and perceptual system to work together is something that we must know an enormous amount about. So that's, where's our time, 19 minutes, so great. So then the next one. Can I talk a little bit about the transparency? Transparency? Oh, yeah, talk about transparency for a second. We've been working with transparency and blur a lot. You saw it in that 30-layer example at the beginning. And we're beginning to build it now into the three-dimensional environment. So, David, why don't you show? So we're beginning to be able to blur out information through transparency, bring forward important information. You can pop it off and on, obviously, through layers, but it will be important to be able to, in real time, clear up information, change your point of view, shift it from one sort of abstract map to a realistic map to, perhaps, a diagrammatic map. And this environment is going to allow us to do that, and I think much, much more. So we're going to... To the shuttle. The shuttle. So this is a visualization by another, one of our students who has been working with NINEX, who is one of our sponsors, to visualize their trial shuttle. It has two hubs. And again, this is the two-dimensional representation of it. And it is, however, a three-dimensional model. And so it's clear that you can see certain forms of things in two dimensions, and three dimensions allows you this incredible facility of metamorphosis and transformation. So each one of these is, when it's working in real time and it's hooked to a network, is live. And every one of these spokes is some form of a connector. So you're able to see, as you can see, a lot of information in a very compact space. So there's a kind of compression that isn't really computer compression so much as this graphical compression. So each one of these represents and will be a kind of real-time video feed. You're able to query each one of these sort of symbols, which is a different kind of space, and find out whether somebody's there or whether they're not or whether the line is busy and do some troubleshooting and so forth. This is a logical view, and you can also, in this case, look at a physical view and transform from this to an example of a building and place people within their offices and then be able to understand very different kinds of things and go back and forth. And of course, this is just one kind of transformation and we're looking forward to doing studies of lots more. So again, we don't have a video board yet, so there isn't, oops, this is a small simulation of having a moving video in context with all these other things. And of course we would have voice and gesture and so forth. But these are the things that adhere to our heart. So I think that's, I think we ran most of it. 22 minutes. So we want to move from here. Yeah, there are a few little somersaults that are going to have to be dealt with. These transitional moves are really extremely interesting. This is an example of presenting, trying to use intersecting claims to represent complex information, or in this case, it's mutual fund data. It is not up to date, but again, it would be connected in real time. It shows six funds. And I think seven kinds of features about the funds, including David Liston, because I can't read them, but there's risk and there's return and so forth. If we could show the funds themselves and you get there from here. So this is done by one of our students who had architectural background and who had been before she came to us doing, okay, so here's your portfolio statistics and you can see up there the number of funds that you're comparing. And you can chug through each feature of information about the funds. And then you can chug through funds in the X space and Y space, I always get confused because I'm not sure exactly where we are. But what I think is obvious about this kind of a construct is that you can bring together information and see it contextually in a way, a portfolio is a kind of good model because you wanna see how things are affecting each other. And that's a very difficult thing to do right now. In any environment, people do it a lot in their heads, but we're looking for graphical models that will allow us to amplify that part of the cognitive process. So in addition to just getting numerical feedback and relationships in the X and Y coordinates, we are able then to pull up bar graphs and chug through time over a period of roughly 10 years for each fund. So you're going through, you're checking annual return against risk. This data came from Morningstar. And so the upper bar graphs will go from what, 1983 to 1992. Unfortunately, the lower thing isn't working yet, but you can visualize a one year, three year, five year, and 10 year that's coupled to the annual return. And those are the kinds of things that you begin to wanna be able to see, not just for one fund, but multiple ones. And it's an extraordinarily interesting challenge. This is 1990, which those of you who have any money in the stock market will understand, oh well, it's just something I've recently learned about. And this is the beginnings of a mapping surface of the time, the temporal history. So we've got 26 minutes in, we probably shouldn't show the other two, right? We weren't able to get them integrated into this for technical reasons, but they're interesting. So the other two have to do with the internet. So be patient while they come up, okay? They're not as seamless and as interesting, I think, but it takes a couple of minutes. I'm trying to think what I should say while I'm waiting. While we're waiting. Okay, that was fast, good for you. Okay, can we have the lights down again, or? Thank you. Okay, this is a beginning idea. And I think it's fascinating and interesting enough to show, even though it really isn't working very well right now. So this has gone out, it will go slow for a second. This has gone out to the, in this case, clarinet, but it could go anywhere, you know, it could go to the internet, it could go to middle gate, wherever. And it's gathered through key words. And key words are really dumb. And there's a lot of very interesting work that we have been collaborating, not collaborating with, but taking advantage of in the rest of the Media Lab, one of which is the agent work of Patty Mas and another is the work done by Professor Ken Haas in relational parsing. And so we expect to have much more sophisticated ways of searching these massive amounts of information. And once you're searching it, you really need some incredibly interesting and coherent way of presenting this stuff. It's not enough just to find it because you just can't make your way through it. And I think Nicholas's statistic yesterday about the proliferation of the use of mosaic indicates how many people really need something to allow themselves to traverse and browse this stuff. Well, this is bringing forward sort of main categories. It's searched, I don't know how much data, not a hell of a lot, but what you can do now is chug through this grid of categories and if you move from government to broadcast, I don't know whether you can... Sure. Yeah. These little radiating red lines tell you how many connections there are to articles based on the key words and the direction in which they're going. So there's a set of algorithms that are working that are going to be able to wait things for you and give you some graphical significance as to what it is or at least the amounts in this case of information. Can you show a couple? Can you go through a couple of others? Should I go in? You mean? I'm afraid to go in. I'm afraid to go in. It's gonna break, yes. Anyway, but you see what the idea is. The idea is to then go into subcategories and from the categories which are clustering as it browses the database, then you move into the actual articles and their headlines and then from the headlines you can move into the article itself. Well, you can see all the problems that have to be solved but it's pretty damn interesting. The girl will be very happy when I tell her. Anyway, there you go. But these are the things that we need to research but I think it's terribly exciting. He's also using a centrifugal and centripetal force to bring things close to the center. So let's give you a final thing by Yin-Yin Wang. That visualizes the news from different points of view. And so what we need to think of is this massive information of the internet and there's a nice little, I think, metaphor. Maybe it's not a metaphor. But the difference between going into a bookstore and browsing and the fact of reading are two very different but very, very critical ways of gathering information. So this comes closer to trying, okay, so the colors are here are very subtle and they're not translating very well but picture below the internet and some version of what you saw previously to this. And then coming on a plane which in this case is foresighted and it's only one. Can you identify each side and pulling out by category things that you're information. So this is headlines and then sources and length of article and the agents ordering of the articles. So this is assuming filtering and a lot of personal knowledge about what you're interested in. What's absolutely intriguing to me is that you don't lose your context and I want to point out these little cues that don't read very well right now but they will, there's a lot of problems with typography in the SGI. Can you shut, where did they go? Yeah, there they are. So you see the category of entertainment right there and these anchors I think help you to move between three and two dimensions. So we go quickly to two dimensions and say goodbye. So you go to the top and now you can read.