 Let me now start to give a high level overview of what we are going to be covering and what the goals are for the course. So, I will start with the goals um there was going to be some kind of cloth for a racing, but there is not now. So, I am not sure what they want. So, I will use this. So, let me just write out the goals. I will spend some time talking sometimes writing somewhat slower to write, but at least for some of the more important things I like to write them down. So, some goals for this class learn to build a good maybe great virtual reality experience. Now of course, this changes on the definition of good. So, I will say a little bit about that in a bit, but let me state some more goals understand we are at a university here our goal should be to understand. So, understand how virtual reality works learn to criticize the R right. So, you should be very critical I want to do my best to prepare you to criticize that do not just put on a virtual reality headset and go wow this is the coolest thing I have ever seen it may very well be perhaps, but learn to be critical so that we can all improve the experiences. So, that is one of the things I want to help train you on and finally, learn fundamentals to shape or build the future of VR. So, in virtual reality the the technology is evolving very quickly. So, if you want to keep on top of that you have to read reddit posts almost daily look around throughout the internet keep track of what is going on in the media companies are racing to make consumer products and they are going to keep on improving very very rapidly. I want to focus on the fundamentals I want to try to give you the concepts that should be more invariant with respect to time right. So, give you some basic truths and give you some basic concepts. So, that you can help build the future of virtual reality and have some knowledge that hopefully does not become stale in one year right. So, that is something I am focusing on. Now, the lab experience that you are going to have will be exactly using the latest technology and some of those experiences may become obsolete in a year you may use different types of software tools different types of headsets and things like that. So, the lab component in some sense is going to be time stamped to what we are able to do in the summer of 2015, but I want these fundamentals to go into your heads as well and give you some perspective and understanding that will survive the test of time. Because many of these kinds of concepts in virtual reality have been around for decades people have understood them for decades our understanding continues to improve and I want to give you the snapshot of at least my understanding as I consider both the intersection of the technology or the combination of the technology with these fundamentals in today's world. So, as far as good what do I mean by good well I will say that it is comfortable plus adequate I know adequate does not sound very demanding adequate for the task. So, by comfortable I mean it should not make you nauseated it should not give you headaches fatigue things like that. So, I want to try to understand what makes an experience comfortable and it should be adequate or sufficient for whatever the task might be. So, what might the task be maybe the task is to develop an immersive first person shooter video game right. So, maybe that is the task maybe the task is to give you the feeling of virtual travel you may feel like you are in another part of the world maybe live exploring into the part of the world maybe you want to maintain close relationships with your friends or your family while being far away right. We try to do that with a phones or a video conferencing Skype things like that maybe we can take that to the next level. Maybe you simply want to relax watch a film a movie of some kind maybe the movie is interactive maybe it is not. So, these are the kinds of tasks we want to pick a task and then try to understand what kind of level of quality let us say or what types of things do we need to worry about for that for it to be adequate. So, that you succeed in the task you may feel like you are virtually traveling and it works wonderfully and your as a minimum baseline it is comfortable right. I am going to now give a definition of virtual reality I feel like it is important to give definitions from time to time. So, let me do that and then we will talk about this. I have seen quite a few definitions around very often they seem technology dependent. I am going to try to give one that looks more fundamental let us say and perhaps a bit surprising. So, I will call let us see definition of V R inducing I have a lot of writing here to do apologize for that inducing targeted behavior in an organism using the word organism may or may not be human. So, pay attention to that by using artificial here is where the technology comes in sensory stimulation might be visual might be audio could involve touch smell taste all kinds of things sensory stimulation while the organism little or no awareness of the interference. How is that? So, in some sense the organism is not consciously aware that it is being fooled if consciousness is appropriate for that level of organism. So, you are not quite aware of being fooled at the moment if you are human you might remember putting on a virtual reality headset at some point, but while you are in the experience it is very easy to forget that and believe that you are somewhere else. So, that is the idea artificial the artificial part is where the technology comes in you are somehow taking over the senses you are hijacking the ordinary sensory input that you get and replacing it with something that is been engineered and it is fooling the organism. So, that is something that is very important to take into account and you know when I think of a V R system if I draw kind of a blob and I call this a V R system let us say V R system I like to think that it is splits into two halves there is the human part or I will say organism or generally to fit my definition and then this engineered part engineered hardware and software right. So, this is coming together it is very easy for those of you who are trained as engineering and computer science students to just think about this engineered part right just think about the engineered part. However, it is very important to remember that this part is here as well the organism part is here as well. So, that when you try virtual reality you become part of the system and it is very easy to forget that. Something really fascinating that has happened lately is that virtual reality has become popular in neuroscience, biology or we could say among neuro biologists and perceptual psychologists. Did you know that they use V R systems extensively in their research? The reason why they do that is so that they can study organisms they can present a controlled stimulus to an organism and see what the effects are. So, I have seen virtual reality applied to rodents say laboratory rats mice gerbils I have seen it down to bugs cockroaches all the way down to even drosophila or fruit flies. They got they get them to run on little tiny spherical treadmills about 3 millimeters in diameter and they have using some chemical processes they can essentially turn off some of their neurons and then present visual stimulation to them and get them to run in different ways through the visual stimulation and see the effect of changing their neurons and study the effects of that which I really find fascinating. So, for them virtual reality is about as powerful as say an MRI machine to a doctor you know it is some way to measure the human body and we work most of us here probably work in forward engineering let us say if you work in neuroscience and biology and perceptual psychology it is essentially working in reverse engineering right. The system has already been built and you have to poke it and prod it and manipulate it and try to figure out how it is working right with the goal to understand. So, I found that fascinating and that fits very well with this definition of virtual reality they have they exactly see it like this and that is in fact why they love this technology and that field actually has quite a bit to contribute. I want to show you a few pictures related to that this almost looks like my definition of virtual reality. Imagine a gerbil or a hamster I will call it a gerbil. So, imagine a gerbil running on a wheel this is at least in terms of your motor skills and motor activity giving some kind of feedback. So, that the gerbil feels like it is running and humans can be fooled in the same way I guess this one is actuated with a motor I suppose if you had a low friction wheel you could get a human to run on that just fine as well to match the gerbil. This is an example of a virtual reality applied to a cockroach so it runs on a foam ball that suspended using air to make it nearly frictionless and there is a wire there to study to do single unit recordings of its neurons and then it is presented with scenes visually on a screen and so it will start running and exploring and you can control and manipulate it in some way this was done at the University of Olu in Finland. I had one more picture, but it seemed to have disappeared that is right. This is a picture of humans more or less doing the same thing as the cockroach right. So, here they put on a virtual reality headset, but you could also have the surround screen if you like and they are running around in some kind of omnidirectional treadmill dish which is which was produced by a start up company recently. So, it is very interesting to me to see humans and let us say cockroaches and other lower life forms fooled in the same way right. So, that is just something to pay attention to. There is one very interesting distinction which is that for a scientist they have a very clear distinction between them and the subject right. There is the scientist and let us say the laboratory rat right. Now, when you are the developer of virtual reality systems you become both a scientist and the laboratory rat if you are not careful right. So, you are doing the experiment on yourself while continuing to develop it. So, that means you are giving yourself inappropriate feedback because you are fooling your brain while developing it eventually you fool yourself into thinking you have made a wonderful virtual reality experience, but you are strongly biased because you might like what you have created right. So, it is very very important to get feedback from other people especially from subjects who have maybe never been exposed to virtual reality, especially ones who have not been playing first person shooter games since they were 6 years old or something right. So, it is very important to get feedback from other people and also to train yourself to try to look for flaws in the system and I will get into some of the details of that. But that is just something very different about let us say neuroscience applications of VR very clear distinction between the scientist and the subject of VR developer is both at the same time they flip back and forth without even being aware of it. So, I think that is an interesting thing that happens. So, other terms that I want to talk about we often use words like immersion the feeling that you are surrounded by the sensory stimulation. So, you may feel immersed by wearing earphones and you have an immersive audio experience. You may go to an IMAX theater or some kind of panoramic movie theater and you feel immersed in that sense. So, immersion stronger word is presence where people like to say informally that your brain has been tricked and you feel like you are somewhere else completely and tend to forget about the fact that you are not. And when I first heard that I thought that was I do not know kind of a fuzzy concept I was not so sure what I could do with that it does not sound like an engineering kind of concept to me all your brain is fooled. Well, I recently learned that this can be established scientifically there is something called place cells in your brains and in the brains of rodents where there is particular neurons that fire whenever you are in a particular place. So, they put for example mice in a maze this is a simulated picture of a maze here and they have noticed that by doing what are called single unit recordings just by studying the neural activity in particular places they have observed that when the mouse is in a certain location a single neuron will start firing that corresponds to a code for that place. And they have done virtual reality experiments where they have shown that these place cells are learned even for virtual environments. So, I think that is a pretty concrete definition of fooling your brain your brain will start to learn three dimensional environments places inside that you have never really been to. They may be real places that you have not actually been to or they may be virtual places that are completely fictitious, but your brain is fooled by that. So, I think that is pretty a reasonable concept presence. It is another interesting concept which is and this one is used by the neuroscientists. There is open loop versus closed loop. Those of you with control theory background may recognize this distinction. So, in the open loop sense the stimulation is just timed. The sensory stimulation provided this artificial sensory stimulation provided is simply timed it does not matter what you do. So, if you are sitting in a panoramic theater watching I do not know maybe your favorite Rajnikanth movie let us say alright everything is just coming right along you are not really participating in the action. You might kind of feel like you are there it might be immersive. You may feel even a sense of presence, but it is open loop. If you actually get to interact maybe you get to fight with the bad guys right and actually change the outcome then this would be closed loop. And scientists make a big deal out of this because the organism needs to interact with the environment in most of the simulations that they do. So, I attended a workshop at Ludwig Maximilian University in Munich of about 2 or 3 weeks ago and gave a keynote there about virtual reality. I learned a lot about the science of neurobiology and virtual reality applied to that. There were about a half a dozen labs working on these kinds of things and part of the title of the workshop included the phrase closed loop. So, it was closed loop virtual reality which I thought was very nice. So, more words and then we will move on to historical perspective. So, telepresence or teleoperation operation. I have worked for a couple of decades as a robotics researcher. So, I cannot help imparting some robotics perspective on this as well. So, imagine having cameras connected to a robot could be a flying robot, a swimming robot, a rolling robot, a humanoid it does not matter really the kind of robot, but you can use virtual reality interfaces to feel like you are somewhere else. So, the reality itself it may be real reality, but you are simply not there right you have been transported. So, that is just something to pay attention to. The experience you have may correspond to something quite physical and quite real and it has been captured by cameras in some way maybe 3D cameras, maybe 2D cameras and it is fooled your brain and that may be just quite fine. And it does not have to be a perfect cartoon like world that has been created like in a three dimensional video game right. So, it can be either kind and so it is important to consider these kinds of applications. Teleoperation being an important case of that as well where you interact with that world. And issues of latency or delays become very important there are a lot of interesting applications from aerospace to medicine and tele-surgery and things of that nature lot of interesting research going on here in that subject for example. So, that is something to keep in mind. I think I think one thing to think about as well in terms of this definition that I gave is think about what does VR include? I skipped over one word that I meant to put here. Let us say augmented reality augmented reality. I will consider that as a kind of special case of very important kind of special case of virtual reality. So, it fits well within the big picture. In this case very similar to telepresence and tele-operation the actual physical world is being strongly emphasized. In fact in augmented reality your brain mostly believes that you in the same place where you've always been standing everything is fine. However some additional information is being super imposed into the world. And if you take it in the limiting case maybe you just super impose so much information that you cannot remember the world where you're at anymore and then you get you feel like you've been completely teleported right. So, these are very into a virtual world. So, these are very closely related. So, I wanted to put those things together. So, I would include that in virtual reality. What else does it include? Let me put some maybe absurd sounding things. Let's see well let's just try watching a movie. I don't know would that be virtual reality? No answers here. No, yes I don't know. There's no clear answers here. I don't know just something to think about. I want to give you something to think about. Video conferencing, conferencing. What about listening to music? Right? You put on your earphones talking on the phone talking on the phone really. Have you ever been immersed in a conversation with someone and then maybe fall into a pit or something as you're not paying attention to what's going on around you. So, we could say just playing a video game. This leads me to the next comment which is very important which is think about the difference between being first person versus third person. Right? Do you feel like everything is happening to you all around you or do you feel like you're watching it happen? Right? Yes. I'm not sure. Well that's very interesting right. So, in terms of the audio sense, so one thing I should point out is that our visual sense is the most powerful, the most complex that we have. The largest fraction of our brain is devoted to that vastly compared to the other senses. So, I really think it's quite equivalent when you just put on earphones and you listen to music. You have the sense that there's a band either inside of your head or all around you depending on how well it's been recorded and reproduced and that gives you a feeling of presence in the music somehow. I don't know. I don't think these are, you know, I'm trying to intentionally pick some things that are on the boundary to get people to think of it but I'm not going to go to the sort of crazy obsessive level of just saying, okay, by the time this course is done you'll believe that everything is virtual reality including your everyday life all the time. Right? So, I'm not going to make a question. Oh, perfect. Here's a good one. So, this one I'll finish with because it's probably maybe the craziest reading a book. Right? If you're reading a really good story do you feel present somewhere else? You completely lose yourself. You forget your surroundings. Wow. How can it be happening, right? So, that kind of virtual reality has been around for quite a long time. All right. Any other questions for comments? So, I just tried to solve a problem. So, I closed everything and just focused on that problem. So, does that constitute a virtual reality? Oh, very interesting, right? So, maybe a really difficult math problem, right? I'm sitting down writing and I've blocked out everything else. I'm doing some kind of interaction with a pencil and paper. Very interesting. I don't know. That seems like another level yet, right? So, if you're sitting and drawing nice diagrams and you're imagining you're in some other place where the diagrams are, that may work. If it's simply mathematical equations, I don't know. It's definitely a fuzzy place where I'm afraid to answer. I'm not sure. I feel like in the case of reading the book, I start to imagine a scene. I imagine some other people's lives. I feel like I'm there somehow, right? I may be in Victorian England or something all of a sudden, you know, reading Charles Dickens and I feel like I'm there. It's a little different than what I get from solving math puzzles. But there is that sense of blocking out all the other sensory stimulation, which is attention and focusing on one thing. I think that's right. So, I think definitely this, let's say selective attention is an important part of it here, but I feel like there's also something else, like the kind of belief that you're immersed in some other culture, place, people's lives, something. That's another part of it.