 topic I have which is depth perception. I will go a little bit into that. This is covered in chapter 10 of the of the Mather book. Depth perceptions and it is a very important topic for a virtual reality right. So, we like to think that one of the biggest differences in looking at a screen compared to looking through a head mounted display is that we can provide to you depth perception through stereo, but not completely true that that is the most important distinguishing feature. We get depth information from many, many sources and I want to make sure that is very clear. So, depth perception. This is by the way a very general pattern in perceptual psychology is there is something perception. So, we have depth perception we will talk next about motion perception. There is some other things I will not have a lot of time to talk about but we could talk about scale perception we can talk about color perception right. So, this is just a template here if you like. So, if you want to study more of these things later it will follow the same mentality. So, we will be studying what are called depth cues and again in this pattern cues over and over and comes up again. So, if I if I am studying motion perception then I will talk about motion cues. So, cues are somehow pieces of information that trigger the brain to perceive whatever it is that we are trying to perceive right. What is what is the key information that we need? What are these features that are going to be used? It is going to be two different kinds of depth cues. One nice way to separate them is into what I will call metric and ordinal. In other words metric is going to be varying continuously with distance and ordinal is about ordering as the name suggests for example, near to far. So, what is in front of what else? Another way to maybe name these number one could be maybe continuous if you like and number two could be combinatorial right. It could appeal to the computer scientist in you if you have that background. So, again the thing I want to emphasize is that there is a multiplicity of depth cues right. So, not just stereo. When you look at panoramic images for example, if you have had a chance to look at those in the lab they may look quite three dimensional even though the same image is being presented to both of your eyes. So, why is that right? So, we need to talk about that. All right. So, I will go through some examples one retinal image size. So, how far away it is may have to do with how much of your retina the image takes up. Combined with your knowledge about what it is you are looking at right. So, assuming you are looking at something familiar. Let me give some pictures here. So, for example just based on the size of that hand as an image on your retina in the context of a person standing there behind you make some inferences about how close right. Does the woman have an enormous hand in the image on the left or is it just closer? You do not see an arm there really it is the same picture either way right. So, you are making some inference about depth and you are not using stereo. Again, what is the size of the image in the retina? So, we know maybe how large a tree should be and so, if we see two trees and one is further away. So, on the right there it shows that the smaller tree is perceived to be further away simply because it occupies a smaller region of the retina. Another thing we can look at is height in the visual field. So, let me write that height in visual field. So, we did not change the size of the person there we just changed their height in the visual field there is a horizon line and so, it appears that the the man at the top is further away. So, that kind of information we are also using. Here is another one where is the spear going to hit right. I guess there is also some size changing there as well actually combination of two of them I would say because there is height in the visual field, but there is also differences I do not think the elephant should be smaller than the animal in the front. Texture gradients and perspective, texture gradients and perspective is a part of this may not be a perfect texture. So, we have some examples of that you perceive depth right just from the from the the the street stones in this painting. It is a lot of perspective near the top there in fact, it looks a bit excessive, but but at least from the the tiles on the ground you can see you get some idea of depth you get ideas of depth from this right just an arrangement of lily pads. So, again depth from this kind of texture gradient again is a kind of texture here texture gradient. Another one is image blur. So, it looks in this one that the blurry parts are farther away right. So, further away just by the way we have drawn this image we perceive the pink flower to be closer. In this case we have blurring up in the front we perceive the garden gnome to be further away. This is a case before. Another one is atmospheric, atmospheric perspective. So, the hazy mountains in the distance right seem to be further away. So, it is some additional q. This one is great shadows let us see 6 shadows are shading. So, in the top picture I perceive the balls to be going further and further away right as we go from left to right, but in the bottom picture I prefer them I I perceive them to be at the same depth, but just different heights in the air right. And the only difference between the top and the bottom is how the shadows are rendered. So, clearly you are using some additional information from the shadows to help you reason about depth it is happening all the time for us. You can also look at these shadings to figure out whether it is a cylindrical shape just from the way the shading works here. So, in addition to just straight forward shadows there is a just shading across the object. It is the one on the left an inward cylinder or outward cylinder I am not sure we can tell there may be ambiguities there looks like the other two are a little bit clearer maybe not sure you know. So, maybe the last one looks clear that one looks outward right. So, be careful with those, but you do get some depth information. Another one is interposition right. So, I perceive the yellow disk to be closer than the red square which is closer than the green triangle. So, we get some kind of depth ordering that is an example of ordinal information and the same kind of thing is happening in this complex picture right. So, there is a lot of boats out there people in the front waving, but we get a lot of information just from the ordering and not necessarily using any of the other queues in this particular case there is not a lot of extra information there, but we start to infer about various sailboats and where they are located with respect to each other from this interposition. So, let me give a few more here goes on and on does not it 8 accommodation and this we talked about before right is this refocusing of your eye. So, just based on your brain's knowledge that your eye has had to refocus you perceive something as being closer. Now, if you mess that up with your head mounted display by making everything appear to be at infinity all the time you have lost this queue. What is the effect of that? I do not know someone should study it, people have studied it, but there is still a lot of unknowns. So, you are losing that queue many people would argue that is a very important queue to keep for depth maybe more important than stereo. Motion parallax motion parallax. So, if I move back and forth I guess I need to hold some kind of object stationary here, but if I hold the station I go back and forth then I see this object passing in front of you who are further away at a faster rate right. So, the objects at varying depths intermingle in a certain way where the closer objects are moving more quickly as I move back and forth right very important motion parallax that is important for visual sense it is important for your audio sense. So, it shows up in other places. So, it is important here as well. Now, if I have a head tracking algorithm that is running with sensors and such for a head mounted display if I can only rotate I get some motion parallax if I simulate the translation of my pupils still using rotation, but if I move like this I will not get motion parallax. So, if I look at a panoramic display and it was taken by a panoramic camera at some fixed location and then I start doing to do these parallax motions I will not get this beautiful arrangement of objects moving back and forth. So, you will lose that if you capture a panorama with a stationary camera right. So, you lose that bit of information. So, it is very important to have that, but that gets lost in that case and finally, number 10 I save this for the end the obvious one which is a stereo stereo cues. So, what are these there is the vergence angle right. So, your brain knows how your eyes are oriented how much they have converged. So, you have a signal for that you have what is called binocular disparity. So, how much how different are the images between the two eyes now there will not be because your eyes are rotating it will not be the same as if I just had two cameras facing forward and then I move them apart to get stereo. I am not rotating the cameras to face the object like I would my eyes actually rotate to face the object of interest. Your cameras would not right unless you put some special motors on them to rotate them. So, be a little bit careful. So, in doing stereo computer vision you get a very large displacement when you when we are talking about the human eyes they are rotating and converging to fixate on the object of interest, but there is still some disparity in the images right even though they cannot look identical because you are looking at them from two different perspectives. So, that is what I mean by this binocular disparity it is not as much as it is in the engineered case, but nevertheless your brain can detect that information. There is diplopia which I mentioned earlier if I fixate on some close object then there are multiple images in the periphery that is additional information. A couple of more things I will say about stereo you have to pay very close attention to what is called the inter pupillary distance or IPD. The IPD in the real world becomes very important if you are going to place lenses in front of your eyes right then you have to line them up correctly so that your eyes are centered when you are looking forward that is the best you can do. If you do not get that right if you do not have adjustments for this do we have adjustments for those in the lab does not look like it. So, if your IPD is far from the average which you might not even know this what your IPD is most people do not know this. My IPD happens to be lower than most. So, I am I can remember maybe the 15th percentile or 10th percentile or something. So, my eyes are actually close together you know who would have thought I have no you know compared to the world average. So, so if you so that is one question is are you looking through the center of the lenses that is very important for the optical part of head mounted display. Another interesting question is in virtual reality now have I matched you know if you believe you are in a perfectly scaled world have I matched the inter pupillary distance perfectly or is it too small or too large. If you make it very very large in virtual reality you will feel like like a like a like a like a big monster or something right you may look like Godzilla or something. So, you can make an entire city look very small or feel very small you move your head around and and you know looks like it has been miniaturized right. If you make your IPD very small you might feel like you have become tiny right. So, there are some there are some applications people have written where you can make yourself feel like you are you are suddenly maybe 10 centimeters high right and so part of the way to achieve this is by decreasing the inter pupillary distance. So, that definitely has a lot to do with your perception of depth and your perception of scale. Let us see here this figure just shows what is called the horopter which is the region when you fixate it is the it is the stereo region that you get. So, a common stereo focal region when you consider the clear images projected onto the retina there is you know remember we did all these study of focal planes this is showing the stereo focal surface that is common for your two eyes. There is the calculated theoretical surface for that and then there is what has been determined empirically which does not seem to match exactly. I personally do not know why they do not match. So, I do not know what theories there are for that, but that is what we get and this may also provide some motivation for people advertising curve displays perhaps. If you sit exactly in the right place then it may give you a perfect perception, but you know who knows. Let us see are there any questions about anything I have gone over today. I will leave you with one final optical illusion just for final surprises, but see if you have enough background to explain why this might be occurring. Does that look strange to you? So, this is the it is called the cafe tile illusion. All of those lines are horizontal right back wing back and forth, but they seem to be bending don't they, but they are not. So, I will leave you with one optical illusion there to think about. So, that is it for today. Thanks.