 So, last time we had this alternate world generator maybe I can just abbreviate it this time as AWG alternate world generator and we have um let us say an input side get this out of the way here. We have an input side and an output side and on the input side we get information from let us say a tracker um let us say maybe just a head tracker head tracker if I can make that go into this AWG um maybe a game controller it does not matter what in particular I am putting in here I just want to give you kind of a high level view of the um of the system um keyboard maybe I can put a hand tracker in whatever else. So, there is information going into this the alternate world generator has to use this information to make some kind of updates to the renderers. So, for output see there could be a um let me write it first um visual renderer and then eventually this goes out to a visual display and we talked about different examples of that and it does not really matter which sense. So, we can have another one as an audio renderer audio renderer we get an audio display and I want this to continue onward for other senses. So, if you want to have a haptic renderer you can have a haptic display that gives you force feedback if you like right. So, the job of the alternate world generator is to maintain this kind of fictitious or remote or recorded whatever kind of world it is it has to kind of maintain a lot of the physics of that world a lot of the geometry and physics of that world in such a way so that you can output to these renderers which then output to the displays which then output to your sense organs which then fool your brain right. So, that is the idea. So, we have to figure out what goes into this alternate world generator has the main part of the software challenge here. So, here is another way to look at it if we had a little bit of geometry I can say imagine we are super imposing alternate and not really sure what to call the world that you are actually in. So, the alternate world and the local world maybe I will say local real world kind of emphasizing local the non alternate world all right. So, think about superimposing a. So, somehow there is an alternate world that is being generated by the alternate world generator I want this to be really geometry here. So, there is an alternate world and in this world there are you know maybe furniture around pieces of furniture around obstacles of some kind right. So, there is stuff in the world in this alternate world and then this kind of story is being maintained that is fooling your brain that you are being placed into this world. So, somewhere you appear in this world. So, you maybe have your own kind of region where you are able to move around. So, your head has been transformed into this world embedded into this alternate world. So, your head is here and you have your eyes looking out in some direction here and they are kind of like virtual cameras looking out and as you take your head here and move around there should be a kind of perfect correspondence you are moving around inside of here. This yellow zone is where there is a kind of super position or overlay between the local real world and this alternate world or kind of virtual world that you may be constructing could be virtual could be real, but it is the alternate world. So, there should be a kind of overlay here between the two and there is interesting things happen if for example, you are moving your head and all of a sudden maybe this is one of the obstacles here this should be a top down view. So, all of a sudden there is an obstacle in the world some big piece of furniture or wall and you decide to move your head into it what should happen? If you have haptic feedback it could smack you on the head right. If you do not what is supposed to happen? It is a difficult problem. So, you have to figure these things out right or maybe you just want to grab on to the controller and move your character around move your virtual self around. If you do that what happens when you come to these? You need collision detection you need something that will tell you know you cannot go that way or maybe you want to move right through it that is the physics of this alternate world you have to decide on what the physics of this world is going to be based on your task and based on things like comfort. It might not be very comfortable to just start passing through walls all of a sudden or it may not be comfortable to just have your virtual self abruptly stop in the virtual world because there is a wall that may not be comfortable. It is difficult problem and I will help you to understand why that is as we go along. In fact, I started talking about motion. So, imagine this yellow rectangle that is like your zone in the virtual reality system you may be you may be sitting down in a chair you have the headset on you can move around like this right if you are on a rotating chair you could rotate around. So, you are moving around inside the virtual world, but in the physical world you are also moving around that is this yellow zone here and maybe you cannot leave the yellow zone in the physical world. If you can then you have a larger super position problem right maybe you are in an enormous region in the physical world and you can walk around in there with a headset on you are being tracked that is fine you have to deal with again this whole problem of if you can really walk around. So, this yellow region is larger then you could interact with all kinds of obstacles it ends up being an interesting challenge right maybe a better experience overall, but you have more challenges. So, I want to talk about different kinds of self motions and then we will get on to the sensation and perception part which is the final part of the bird's eye view. So, self motion is a very important concept in virtual reality called ego motion or self motion. So, there are two kinds of self motion. One the user moves and the motion is matched in the alternate world right. So, if I turn my head in the real world in this alternate world I am also turning my virtual head and the two should match perfectly right. If you do that all with zero latency there is no delays everything is very accurate everything is optically done correctly, audio is done correctly if all those things are done perfectly that should be very comfortable and very convincing to your brain. The other motion is not through an interface an example would be a game controller the user locomotes or moves it is virtual self. So, in this case you are not moving in the physical world you are just grabbing on to a controller and maybe moving a joystick. So, a little bit of motion, but then you are moving your virtual body through space perhaps very quickly. So, this case you can see is quite different right. So, that means you are you are doing it all artificially it is not a perfect correspondence that would correspond in this picture to moving the rectangle around using a controller, but you have convinced your brain that you are actually there. So, that means you are convincing your brain that you are actually accelerating moving along a different velocities and that is where the discomfort part comes from and that is where simulator sickness starts to get really bad. So, to distinguish between the two if you do faithful reproduction of the motions perfect in the local real world and match that to the alternate world you get the number one case here which is considered generally very comfortable and if you do the other case where you grab on to a controller and move then it can be quite uncomfortable and there is a lot of interesting research in how to make it comfortable because you would like to be able to sit in a chair and travel over long distances in virtual reality rather than having to walk there right. So, there is a lot of motivation, but it does cause sickness in many cases if you do not do it well and it is very difficult almost impossible to solve in many cases to make it completely comfortable. So, that is one of the things that people are struggling with. So, case number one to summarize that is where the head is moving inside of this box and case number two is when you are moving the entire box right in inside of this alternate world. Questions about that? Some examples of alternate world generators game engines are the number one example that people are using right. So, we are going to use one game engine in the lab Unity 3D unreal engine is another one. So, game engines great there is other possibilities you can make your own game engine and what if we are own simulator right you can make your own alternate world generator as just a kind of simple simulation environment right you may want to hack it up in python for example you know you do not have to do something that is as complete as a game engine and also keep in mind that if you choose to use a game engine for virtual reality you are abusing the game engine. The game engine was designed to make games on screens and there are virtual reality integrations of those, but fundamentally it took many many years to develop game engines lot of trial and error and now there is being adapted to VR. They were not optimized for VR on a screen you can do this number two motion very easily and nobody complains too much, but in virtual reality people are complaining a lot. So, it was not designed for that the entire experiences need to be redesigned what we would like to have is not a not a game engine, but a VR engine a virtual reality engine. I do not think anybody knows what that is going to look like yet, but that is what is really needed and if we have a virtual reality engine that can support the kinds of things we want to do over and over again in virtual reality experiences will be in very good shape. For the time being we are just leveraging the existence of game engines to make the software easy. So, the game engine is this AWG I talked about the alternate world generator. If you have a robot with a camera and you want to do telepresence then you do not have exactly the situation do you? You do not need a game engine what you need is some way to maintain a panoramic representation while you are out moving around. So, that is different it is not going to be a game engine you probably want to hack something special for it. So, there are a lot of different possibilities and you should really think about when you need to start from scratch or when it is best to just use a game engine. Since this is a two week course we will just use a game engine and make it easy, but if you want to for your project use something much more lower level and optimize it all yourself then I would say go for it you know and get the experience.