 So, I want to do the canonical view transform and then I will be finished with the chain of transformations and then we will change topics significantly otherwise I should say that the distance between the eyes is called interpupillary distance. So, sometimes called IPD ah people who work in vision science researchers in that area just call it PD pupillary distance. This will come up again. So, the canonical view transform. So, to make things reasonably easy to draw for me let me assume that the eye is up here looking straight down. So, you have to reorient with this with using your imagination and. So, there is a straight down and then um we are going to be rendering ultimately onto a display that is rectangular. So, because of that there is going to be a rectangular boundary in terms of the field of view of the eye. For the human eye it is not rectangular right we are engineering people we engineer rectangles more easily than circles and other kinds of curves and things, but um. So, there is a rectangular part to this that is going to begin here and continue through the whole pipeline, but if you ultimately have let us say a virtual reality display that is circular or some other shape then it will show up in here right. So, there is going to be rectangular boundaries coming up. So, imagine the viewing limits for the eye there will be some um let us say down here I want to render a or draw a far plane I will call it. So, this will be called the far plane and I want this line to come right down to and hit the center here and then there will be a near plane put it up here well I better before drawing the near plane let me connect the lines here. So, I want to have the limits of how far I can see please forgive me if my lines get a little curvy let us see here or if they do not quite match, but should be ok here. So, if I take a horizontal cross section I should get a rectangle that is what you are supposed to be looking at through this strange perspective drawing here. And then um the furthest away plane where I am going to consider is called the far plane. So, any objects beyond that I will just consider to be out of bounds they are not visible to the eye. So, beyond the limits of this camera or eye whatever it might be there is also going to be a near plane and anything closer than that is also going to be considered not within the limits. So, I will put it up here let us see if I can get this one right. So, I have another plane here again this center line should be poking right through the center of it hit there there through the center that should be in the center call this to near plane near plane. And then this part that we have gotten here let us see I mean shade this shape here that I get um I will not let me say I do not know what to do about the occlusion here let me uh let me just hit the outer boundaries here. So, I will shade this I will shade this this and this I think I got all the outer lines there. So, I did not mean to color this part let us just leave that here. So, you should be a see a shape there and in out of the pink lines that corresponds to a frustum. Let me try to draw this in another orientation now this picture where I have my i coordinates in the i frame z going away from the i y going up x coming out then I should have some kind of picture like this. Am I really going to get this right without making a mistake probably not and let us see some hidden lines back there as well your brain will play some tricks on you because I have not drawn the hidden lines correctly I suppose I can use dashed lines for the hidden ones you can go back and do that let us see. So, this is the frustum in particular it is called the viewing frustum that is what I was trying to show here in pink viewing is that clear all right. So, that is the geometric piece I have decided on the near plane far plane and I want to figure out now for all of the points of our model in the world that appear inside the viewing frustum I need to figure out where they go inside of this canonical viewing cube this is this canonical cube that remember had coordinates plus or minus 1. So, in order to do that I want to set up a projective a perspective projection model which is going to take the frustum it is going to look at points inside of here and consider the line that goes from each one of these points to the pupil right. So, it is going to extend out like that and you can imagine that the image that I am trying to draw shows up in the near plane here if you like. So, in the near plane you can imagine there is going to be a picture that shows up and it is going to be where for every point that is inside here inside the frustum every point that is inside it is connected by a line to the to the pupil and at some point it strikes the near plane and that is how I get my image. So, if things are very large and close to the near plane they are going to show up as very large in the image, but if they are the same size and then you move it further and further back to the back end of the viewing frustum they are going to look tiny here because all these lines are going to come together and everything shrinks down to a point right. So, every point inside the viewing frustum think of that as having an associated line moving from point space to line space is the basis of something called projective geometry turns out the homogeneous transforms that we did are also related to that, but I will not go into all those fascinating connections here. So, because we are dealing with perspective projection and interpreting points as lines I want to make a very simple picture I want to drop the dimensions down and keep it very simple explain the algebra and then we will handle the complicated case. So, let us try to make a very simple side view of this picture here. So, it will be a two dimensional version. So, here is the focal point or if you like I have been saying the pupil location draw a horizontal line here and a line off at some angle and now I want to consider a point up here P this point is just any point that was in the world right after we applied the over a transformations to get into the world and we then applied the i transform as well. So, it is I guess it is not quite in world coordinates it is now in i coordinates excuse me even though there is been no geometric distortion yet and. So, if that is the case based on this coordinate frame that we have set up I have some distance zp for this point. So, the point I guess I do not really have an x part here because of this projection, but you can imagine it looks like xp, yp, cp and then this height here we move my p yp. So, I want to draw the diagram to look like this and. So, this is again this point is somewhere inside the the frustum and then I will take the near plane up here and suppose it is at a particular location here and it is at some distance d remove let us see now I will draw it up here. So, I will have this focal plane if you like this is the place where I am going to be apparently drawing the image it could be the near plane could be somewhere closer I can move it all the way down and have it very very close to the focal point it is just going to shrink everything, but everything is going to be in the right place right. So, everything that I do all the information is going to be in the right place it just might need to be scaled differently. So, I do not really care where I pick along here too much it is just going to affect the scaling and I can always redo the scaling it is like converting from millimeters to meters or something it is no big deal right very simple. So, the scaling is going to matter too much, but let us just pick this particular plane here right. So, I pick this plane and now I would like to know what is going to be this height right here. So, this amount H let us call it for height. So, one thing to notice by the way I have set everything up here is that we have similar looking triangles we have this smaller triangle here and we have the larger triangle out here right. So, it is like a small triangle contained inside of a larger triangle, but they both have the same angle here correct right. So, I want to look at these are both right triangles and I want to compare these links H and Y. I want to figure out what H is and I want to look at that in terms of or express that in terms of D, Z and Y. Y and Z are already given and D is also going to be given because I have picked some place to make a kind of fictitious image if you like I just need to figure out what H is going to be. So, that it ends up in the right place. Well one thing that is very important to projective geometry these perspective transforms is that ratios are preserved of certain kinds. So, I expect this ratio to be preserved H over D is equal to Y P over Z P right because this triangle on the left is just a small scaled version of the bigger triangle correct right. So, they are only off by scale. So, these ratios of H over D and Y P over Z should be identical. So, if that is the case then this perspective transform applied to Y P should be H equals just solving this equation let us say D over Z P Y P. I have organized it so that it looks like there is this extra term here right that is applying some kind of scaling to Y P. It seems to be shrinking it by some appropriate amount due to these ratios right. So, that is all that is going on here. I just want to do the same trick now, but with more dimensions and I want to make it work inside of my homogeneous transform matrices as well. So, I am going to do I want to increase the dimension and make it work in homogeneous transform matrices yes. This angle here this is the angle you are asking about yeah this corresponds to I guess viewing angle would be appropriate. So, if this point P were in the center of the screen if you want it to be exactly where you are looking then the angle would become 0 and the angle could grow all the way up to some amount that corresponds to the boundary of the viewing frustum. I suppose when I look off on a corner here I have the highest possible angle value this does not have enough dimensions for that. So, maybe I am only looking at the top or bottom which it also could correspond to the corner. So, you could so in 3 D you maximize that angle by going off to the corners that is as far as you can look. If you had a spherically or a circularly symmetric image then it would be the same angle all the way around would be your maximum limit. But as I said we use rectangles in engineering so it gets a little more confusing in higher dimensions. So, yeah good question ok. So, let us see. So, let us suppose I take this transform and I just kind of make it work by abusing our homogeneous transform matrices and I say abusing because I think there is a culture of getting the homogeneous transform matrices to do a lot of work for you even if they might not mathematically always look like they are doing the right thing from a well sorry they are doing the right thing mathematically, but they might not be the most intuitive or natural from a mathematical perspective. And part of the reason for that is that people in graphics are often reusing computational blocks and units they reconfigure them or reappropriate them let us say for another purpose. And so that is partly what is going to go on here and I just want to show that again with a with less dimensions. So, if I want to do this homogeneous transform let us suppose I just want to start with y p and z p as 2 points I am ignoring x here because I have just eliminated it let us say 1 dimension lower and then I will add x in and add everything in a bit. So, if I want to perform this transformation and I want to do it in a homogeneous way I would take this point starting off as ignoring the x coordinate I just have let me see if I am just ignoring the x coordinate I will just have y p z p normally when we do something in a lower dimension we drop the z right, but no I am just doing a 1 dimension lower dropping the x. So, then I write it as a 3 dimensional vector because I am doing homogeneous transformation matrices and I am adding an extra dimension to it. So, the 2 d case becomes 3 d and then I want to calculate exactly this expression here by the following matrix I guess I have room here let us see. So, I will go d 0 0 0 1 0 I am suspicious there may be a mistake let us see y p z p 1 what do I get when I multiply this out let me see here. So, I get d y p p I get z p what do I get for the bottom what is that oh sorry I do not even get I do not even get anything more here right is that right still not right what am I doing wrong here. So, let us see that goes 2 by 1 3 3 2 1 show this is not ok this is correct matrix multiplication at least what kind of abuse we are doing to homogeneous transforms we have not established it right, but at least I think the I think the algebra is ok here I mistakenly put a 1 here do not need that that cannot be right. So, we have this and I want to declare that this is y and h prime and then I will define h equals y sorry up here is y prime is y prime and h prime I will define h is equal to y prime over h prime which when I multiply everything out becomes d y p divided by z p which I think is this expression here. So, what the heck have I done right. So, I am going to use this extra h coordinate this is the extra part of extension here for homogeneous transform where I am going to use the extra row to calculate this h component and then I am going to use that as a kind of normalizer I am going to divide it out in my interpretation of the result. So, I think that is kind of an abuse of the homogeneous matrices, but this is what is commonly done in graphics. So, I am just I am just including it here so that you can get all the transformations you want including the perspective transformation it is not a linear transformation is it right. If you look at z and y as variables is this a linear transformation no right there is a z in the denominator. So, what they are doing is they are using the linear algebra structure here to perform the computations and then interpreting the result as oh I just divide by the last number one I am done. So, they still use the algebra here and then just divide by the h prime is that ok that feels like a hack to me some kind of hack it works I do not like it because you are using linear algebra abusing it to do something non-linear and it causes a lot of confusion I think, but this is done because again in the graphics pipeline it is very convenient for them to just keep performing homogeneous transforms over and over again and they can work to optimize the code that does that very efficiently implemented in circuits and then just continue onward and not worry about the problems of that. So, that is why it is done in this way is that ok. So, that is getting rid of some dimensions I am now going to do the 4 by 4 version here I did the looks like 2 by 3 version right. So, I want to now just expand this out and do the same kind of trick and give you the full matrix and then we will be done with this part. And I am going to change notations just a little bit because I am going to have to have a lot of variables coming in and I want to have it be so that let us see we have that is the z axis and we have this frustum and I want to make this be called n for the near plane this be called f for the far plane. And so now I will give t p I will call it p for perspective it is performing the perspective transform for us it is going to finally, be a 4 by 4 matrix I am going to put n here n is now the replacement for D. So, I am going to n here I will have 0 0 0 I am going to do the same thing now for the second coordinate I am doing it for the x coordinate here I guess it was just y here. So, I am going to use the next coordinate the second coordinate I am going to put it here 0 n 0 0 and this trick of 0 1 0 in order to get this appropriate denominator term this will cause z to carry over I am going to put that here 0 0 0 1 oops wait that one oh sorry 0 0 1 0 sorry 0 0 1 0 I am afraid of the third row here I am not going to talk about it yet. So, so that gives us without specifying the third row you should be able to see that that is just a higher dimensional extension of this trick right. So, when I go and multiply all this out I am going to have the same thing happening now for the x and y coordinates and the x and y coordinates are the main thing that I care about because you are trying to figure out what is going to go on the screen or the display and that is a 2 dimensional thing. And so, you are going to figure out the horizontal and vertical components, but the graphics engine the rendering part is also going to have to figure out the depths and use that information not figure out necessarily, but use the depth information to figure out what should be rendered on top of what like it is one object occluding another it needs to know that it needs to know this kind of information. So, in order to keep that information and preserve it in some way we have to put something else in this matrix. And so, here is what is actually done 0 0 they put n plus f here and minus f n here. So, think of that as the strange row or something unusual is going on. And so, if I multiply everything out here and I try to figure out what the result is suppose I apply T p. So, I do let us see I try to figure this out I do T p applied to x y z 1. And if I apply this what do I get I get you know if we go ahead and do this multiplication I get x prime equals that is transformed x equals n x I get y prime equals you know I am not sure if I am comfortable with what I am doing here exactly let me I guess I need to put some brackets around these to make it look a little bit nicer put one big equals for that. Y prime equals n y and z prime equals something complicated and h prime the last one here it is going to be the same h prime that I have on this for the lower dimension right. So, h prime equals what is that one going to be z and now the z prime ends up being a little more complicated. So, I need to widen my bracket here and it is going to be z n plus f minus f n. So, because of this matrix trick that is being used in order to perform perspective transforms the z coordinate gets let us say somewhat mangled by the algebra. However, if you put exactly these terms in then the z coordinates will at least be correct for points that are either on the near plane or the far plane. So, that is just some algebra if you work out the algebra if you plug in z equals n or z equals f and then you evaluate the expressions where you take you evaluate this expression. So, you divide back out again then you will see that it will preserve the z value in those two planes and for the rest of the values in between they will be incorrect, but the ordering will be correct. So, when something is in front of something else that will be correct, but there will be a distortion. It turns out that distortion does not interfere with the graphics pipeline. So, people in graphics are just fine with that because you will still be able to judge what is in front of what else, but you know do I like it personally not too much because it makes the mathematics not clean makes me unhappy and it comes from the abuse of a linear transform to do something that is non-linear right. So, that causes for me a lot of confusion and headache, but it makes the engineering part more optimized because you reuse these computations in hardware circuitry right. So, that is part of the motivation for that alright. So, is this ok? So, we have that now one more thing we have to do after we perform this transformation what we have done is converted the viewing frustum into a rectangular box, but we have not made the rectangular box have exactly coordinates plus or minus 1 and we have not centered at the origin. So, we have to do a scaling and transform to the box that we have made. So, that is not very difficult it is a very simple scale and transform operation as I said nothing weird going on, but we have to put this into the pipeline of transformations into the chain of transformations. So, if this is what the rectangle looks like by the time we are done the viewing frustum gets converted into a rectangular box. Then I want to say that this corner here will have coordinates L, B and N is still for the near plane L is for left and B is for bottom and then over here let us see I better make sure I have opposite corners here should be up here I have a mistake on my notes, but I think I am going to get it right here R, T, F. So, far plane top and right side of the box right. So, I am just going to use all these letters so that I can put them into the matrix very easily. So, I have one more transform called T I will call it ST for scale and transform. This in Shirley's book is called T Orth if you are looking it up there, but I am instead of emphasizing orthogonality I am going to emphasize it is just a scale and a transform. So, scale transform. So, if I want the coordinates to be plus or minus 1 that means the cube is going to have length 2. So, my scale is going to be for the x coordinate 2 divided by R minus L and then I am not interfering with the rest because I am doing no shearing and then my scale for the y coordinate is going to be 2 over T minus B and then the scale for the Z part is going to be 2 over N minus F and then I am not going to do anything on the bottom of course not doing any strange trickery of abusing homogeneous transforms. And now up here in addition to scaling I have to center this box as well. So, that the translation part is going to be. So, it is just going to be minus R plus L over R minus L and all these look similar I guess the minus is out in front here. So, minus T plus B over T minus B and minus N plus F over N minus F. So, now we have that and so now I can finally put together for you the canonical view transform which is T can which is equal to T S T T P. So, perspective transform is performed first and then the scaling and translation transform is performed. So, that you end up with a pretty box with coordinates plus or minus 1 with respect to the x y and z axes respectively. So, how is that does that seem fine it is nicely separated I think. So, the interesting perspective part is in one chunk this weird kind of math hacks that I am not completely comfortable with that I see that they work are performed to generate this matrix T P and then we have to do the scaling and translation part in order to make a nice box with standardized coordinates. It was not a necessary step, but it makes it convenient for a lot of algorithms at running computer graphics to keep all the coordinates in a nice normalized form. Again some people might want to go from 0 to 1 instead of negative 1 to 1 for all these coordinates in which case that changes this transform a little bit right. The scale will not have twos here in the numerators and these translations will be appropriately changed not too difficult to figure out. Questions about that?