 I would like to now get into um what are called lens aberrations. So, I talked about a simple lens and how it works in an ideal setting and that may be familiar to a lot of you already, but um there are a lot of problems with lenses especially as we increase the field of view. So, if you have a very large lens or let us say if you want the image produced by the lens to be very large um the focal plane to be to be large let us say then there are a lot of imperfections and a lot of problems that arise and in virtual reality we are highly motivated to do that right. If you take a um a virtual reality head mounted display and you put a screen up very close and you put a lens in front of your eyes and you would like to look through the lens at um in in different directions you would like to have a very wide field of view that means that these kind of peripheral effects of lens imperfections are going to become very important. So, they are critical to the engineering of virtual reality systems and that is why I want to go through step by step and talk about different kinds of aberrations right. Aberration means what I mean something um not right something different from what we would like to have happen in an ideal setting. The professional optical engineers deal with aberrations of all sorts right that is their sort of daily bread and butter let us say right of characterizing aberrations trying to compensate for them trying to keep the cost of the lens down in terms of materials and manufacturing um all sorts of issues come into this. I want to give you just an idea of this of the kind of things that can happen. So, I will talk about lens lens aberrations. So, first one I will talk about is spherical aberration. So, the cheapest surface to cut for for lenses is a spherical. However it is not ideal um for for generating a perfect image in the image plane. What tends to happen is the following. So, you have parallel rays of light coming in and then um they do not converge at a common focal point. So, it tends to spread out. So, that means that there is no place where I could suppose I wanted to place my projection screen here and I try moving it back and forth some kind of vertical screen and move it back and forth here. There is no place where I could put the screen so that I would have everything perfectly in focus. If I just use the central part of the lens then it would be good enough, but if I really insist on using all of the lens and we get all the way out to the exterior and that lens perhaps you know ultimately I said it could be a big lens very close to your eye. If we insist on using these extremal parts of the lens then the errors tend to get worse and worse. So, that is one problem um there is a solution. So, so potential solution um you can make what are called ah aspheric lenses right. If the sphere is a problem use something else. So, if you look for the word aspheric and lens you can find all kinds of um engineering literature and research literature on that um from what I have read um the ideal way to fix that is that for the lens you make the incoming surface elliptical and the outgoing lens surface hyperbolic with very carefully chosen parameters obviously. So, that is harder to manufacture than a spherical lens. So, it drives the cost up, but it may reduce spherical aberration significantly and in idealized setting it should not eliminate it all together. But spherical aberration is just one of many problems that a lens designer has to focus on. So, let me give you another one to optical distortion. In this case when we have a high field of view this might happen. If we put um perfectly square grid lines perfectly straight grid lines um in front of the the lens and if you do this for example, if you take the um the head mounted display from the lab and you you can do this you can take out the lens cup and put it over some graph paper it will look something like this right. So, the straight lines become curved this is a kind of distortion called pin cushion distortion and notice that if we were just going to use the lens in the central part here it might be good enough and you might never notice you know this is a pretty extreme example, but you can see that the further you get away from the center the stronger the divergences between being straight and being curved. And since um because people in modern virtual reality systems want the field of view to be as wide as possible they have to deal with this problem and it might not have been a problem maybe 10 or 15 years ago in some other kinds of systems where the field of view is very narrow where it generally looks like you are just staring at a small screen, but if you want to have your entire field of view filled with the stimulus presented by the virtual reality system you are going to have to deal with this kind of curvature. Another common case which is essentially the opposite is opposite or inverse is called barrel distortion the lens ideally is radially symmetric right. So, it has radial symmetry and so these distortions tend to be radially symmetric as well. So, then you can compensate for them by just making a kind of transformation that adjusts the radius in polar coordinates. So, the so the theta part does not have any kind of transformation, but you just kind of perform from the center if you want to compensate for this you just perform some kind of radial stretching and the amount of stretching you do does not depend on theta that is generally how you would compensate for something like this. So, these two are inverses and what is actually done in software. So, I will mention just a little bit this is done in head mounted displays and I will give a little more detail on this later on in the course, but in software barrel distortion and I think I called this t-dist when I just gave you it as an example when I did the chain of transformations barrel distortion is applied to compensate for the pin cushion distortion of a head mounted display lens. So, it is you know I like to think of it as a barrel pin cushion annihilation right. So, the so the pin cushion is one kind of distortion you apply the opposite distortion which is a kind of mathematical inverse and then when you put the two together you get the identity which hopefully should be everything is perfectly straight. It is easier said than done it is very very hard to tune the parameters of your barrel distortion in software so that it compensates perfectly. There are a couple of interesting reasons for that mainly the problem is that a human perception is involved and the human optical system is involved and your eyes move all of this together causes some great trouble in trying to fix this problem which I will give you more understanding of as we go on. Questions about this all right. So, one more I have five aberrations I am going to cover here. So, it is kind of a depressing topic right it is just more it is going to be a list of five things that interfere with the performance of these systems and degrade your virtual reality experience especially if you demand having a high field of view which I think is reasonable to demand. So, another one is chromatic aberration. So, in order to explain this I should start talking about light waves and frequency decompositions of them. So, remember that light one way to look at it is composed of waves varying between about 400 nanometers and about 700 nanometers in wavelength right. So, we gave the formula for converting between frequency and he may remember just a simple formula f equals c over lambda frequency is a speed of light divided by the wavelength. So, you can convert back and forth between these. Now, you have seen the visible spectrum before perhaps right we talk about the basic colors of the visible spectrum right the colors red, orange, yellow, green, blue, indigo. So, when people talk about the spectrum of colors right you have always and you have seen light shining through a prism before I show examples like this in a bit. They are talking about pure sinusoids that correspond to one particular frequency or one particular wavelength right. So, that is a very unusual situation to have it may happen if you generate a light using a particular laser for example, for light that is bouncing around in this room right now there is many, many different wavelengths all propagating at the same time right. So, there is a whole mixture of those and that is called the spectral power of the of the light that is propagating right. So, those of you with a signals and systems background should understand analysis of signals in the Fourier domain and you can talk about the frequency components of that right. So, if you have done some Fourier analysis kinds of things before this should not be surprising it applies to light it applies to sound it applies to all sorts of problems where there are propagating waves. So, that is something to to consider the spectral power of light again in terms of the spectrum I will say it once again when we talk about the visible spectrum and we pick a particular place along it we are just talking about a single blip in terms of this overall spectral power of light. There is one place along the spectrum where there is light at a specific frequency or wavelength there is not a mixture, but more generally there is a mixture. So, spectral power of light you can say that that is like a histogram if you like to think that way it is not necessarily discrete like this, but you may it may helpful to think of it as a continuous way, but but you think of it as kind of a histogram of wavelengths and we are ultimately going to have light going into our eyes through the pupil and hitting the retina. So, what is the spectral power or spectral distribution of that light what does it look like right are there more greens more blues some reds you know how what exactly do we have as we look at the range of wavelengths. So, there is two things that affect that one is going to be the light that is emitted from sources and the second thing is going to affect that is the various materials in the environment how do they reflect the light that is going to affect our perception of the color of something right this board appears to be green and white in some way I think we all see it like that that depends on the properties of the board and the properties of the lights that are shining on me now which as I see them they are mostly white and there is a good reason for that. So, these are the two things. So, I will write them as there is the emissions of the light source. So, for example, maybe I write 400 here and I write 700 here. So, these are in nanometers. So, I want this to be the wavelengths and then one example I will draw these are just rough plot they give you some kind of idea based on reality in some way here. This plot could correspond to daylight this may correspond to an incandescent lamp right. So, we consider the light source and generally when we have a light source if we are going to engineer one if we want to do photography or videography then we would like the light source to contain as much of the spectrum as possible in the visible range and it is also best to have it close to equal you know. So, that it does not overly emphasize let us say red instead of green right. So, you would like it to be what we call white light which would be a perfect balance across the spectrum with the entire visible spectrum represented. When we get off to the extremes down here may be what is that called longer shorter wavelengths is what infrared wait a minute let us see let us see lower frequency longer wavelengths is infrared right. So, that is over here I guess yeah again we can always change using this f equals c over lambda formula. So, I guess based on the way this is drawn this is the infrared part and incandescent bulbs are they tend to get hot right they generate a lot of heat which should generate a lot of infrared radiation which is why it seems to be peaking here would be my guess. So, ultraviolet would be on the other side all right. So, that is one part is the emission of the source and then there is the spectral reflectance of the material. Now, both of these subjects emissions of the source and spectral reflectance become very important in computer graphics when if you want to make a completely artificial scene a kind of virtual scene and you want to render how that might look you need to make models of these things right we make artificial or virtual light sources we will make virtual materials and then decide what its spectral reflectance property should be and then hopefully it will look or convince our brains that it looks reasonable much like it would look in the real world. So, even in computer graphics this is very important, but it is also going to be important just in our understanding of how human vision works in the real world because it is part of this optical system. I am going into all of this explanation because I need to explain chromatic aberration, but this is also going to be useful in many of the things we do in the course here it is useful generally for virtual reality. So, spectral reflectance will be another kind of plot maybe I have at the top of the plot 1 for total reflectance and 0 for no reflectance. So, this is the amount of reflectance maybe I could call it the coefficient of reflectance if you like. So, for example, up here I will still let me write my so I have my wavelengths here and one example I will put along here is snow how is that right. So, snow generally looks very white it reflects pretty much reflects everything very nicely right, but it is not a specular reflection on that when I look at the snow it does not look like a mirror it is a diffuse reflection, but it tends to not look like it is some special color right. So, that is because it has this kind of very ideal spectral reflectance ideal in terms of looking perfectly white or just reflecting back whatever the light sources. If I shine a perfectly red light with only one wavelength in it on the snow what color will the snow be was that green red it will be exactly the light that I shined on it right how it should look ideally you know there may be some distortions and things. If I take something else like say I take a look at grass I realized maybe this is not the idealized color here for grass, but should show up a bit may tend to peak somewhere and it may peak right in the area where that wavelength should correspond to green if the grass is green the grass turns yellow because it is too dry then I guess it will move somewhere else, but it may have a more distinct signature if it is something we perceive to be a particular color. So, that even when you shine white light on it you still get you may still get a very distinctive amount of emphasis along a particular part of this visible spectrum. So, those are the things we perceived as having a particular color you shine a white light on them and not everything is reflected back it is very specific the more specific it is the more we perceive a particular pure color does that make sense all right. So, we have this spectrum now. So, now think about it there is some light sources the light has been bouncing around and it could have multiple inter reflections off of different objects based on the spectral reflectance and how much power gets dissipated remember I said some of the light gets absorbed as well. So, there is less and less as it goes along well by the time it is all done let us suppose now that light decides to hit a lens right. So, if the light hits a lens then here we have a parallel rays coming in and let us do it here off of the extreme you may have seen a picture like this for a prism before. So, as the light comes in it turns out that the speed of propagation of the waves through the medium depends on the frequency right or depends on the wavelength if you like either way which one is going faster here red is going faster through it and blue is tending to get more stuck is that right or is it the other way around right ok. So, red seems to be going faster through here if it were going straight through then it would then there would be essentially no effect right. So, the shorter the wavelengths the slower it goes through the material for the way this picture is drawn and. So, if that is the case then you will have a focal length for pure blue you will have a focal length for pure green you will have focal length for pure red and if you generally have some kind of distribution some distribution that corresponds to the spectral power of light or some histogram of various wavelengths then the focal plane will really be distributed in some kind of way right you may want to put it in one place to really focus the reds well and another place to focus the greens well that does not sound very satisfying right, but that is what you have to deal with what are some potential solutions to that. So, we are still under chromatic aberration I I have shown you a picture there of it possible solutions. So, one find a lens material find and use a lens material with a high what is called Abe number what that means is that it is low dispersion what that means is little dependency well let us see let me put it this way the refractive index n depends little wavelengths. So, there are some materials that refract the same way the speed of light through them does not very much based on the frequency or wavelengths. So, that is one way the trouble with that is that these materials tend to be very expensive. So, in mass produced consumer products this is not very reasonable unless someone finds some new material that is all of a sudden cheap easy to manufacture and all of a sudden saves us. Number two form a compound lens with two materials right two different materials or media. So, this is commonly done in lens design. So, there may be one material which is causing separation the crown glass and then another kind of glass called flint glass which is put right up against the lens to try to bring them back together again. So, you can use two materials play some kind of tricks it is a kind of delicate art for the design of the lens. So, optical engineering becomes very difficult because of these and again if you can get the costs right and the materials correct not too brittle and you know whatever other things you need the optical properties you may be able to make a compound lens like this called an a chromatic doublet and you see even in this picture it does not show it being perfectly compensated, but it may greatly reduce the chromatic aberration. A third trick which would be a computer science kind of solution just fix it in software right and that is being done right now in current head mounted displays. So, compensate just as we talked about the optical distortion the barrel distortion compensating for pin cushion distortion you can also compensate in software by shifting or distorting based on the let us say sub pixel wavelengths. I do not quite want to put pixel there because the if you if you hold a magnifying lens up to a screen which I suggest you do and then you can study the pixel structure very carefully you will see that what we call one pixel in computer graphics actually corresponds to several sub pixel elements that are lighting up right they do not all they do not correspond perfectly they are not it is not such that R, G and B are just all super imposed in one place it is a kind of pattern a tiling pattern. For example, the screens in the lab use a pentile display you may you may observe all right. Questions about this? So, I think that all of them have some kind of flaws the software compensation is not perfect these other solutions are costly and again not perfect it is something that we unfortunately face yes that is a wonderful question yeah yes we do. So, in the human eye we have chromatic aberration, but your brain learns to compensate for it. So, you do not see it and this is one kind of thing that will happen over and over again there are all kinds of problems that our eyes have that our brain is just fixing. The most interesting one was well known one probably being the blind spot that is due to the optic nerve. So, that part of your retina is essentially missing, but do we see a blind spot I can show you some experiments where you can try to find it, but it is interesting, but our brains are repairing all of these flaws. So, that would be analogous to number three here. So, the software can fix it your brain also fixes it no it is very nice question sure. So, that was also these 1 2 and 3s are inside of the big number 3 which is the third aberration which was chromatic aberration. So, I probably should make these you know should have been little little 1 2s and 3s in that sense for the record here those are just inside of my main numberings. So, number 4 is astigmatism which corresponds to elliptical eccentricity of the lens I will just show a quick picture of this. So, instead of having perfect radial symmetry do not do that instead of having perfect radial symmetry when you have light waves propagating in the horizontal plane going through the lens there will be one focal point, but when you have light waves propagating in a vertical plane there will be another focal point. And so, this will mean that there is no place where you can get a perfectly focused 2 dimensional image. Those of you in the audience who have who are wearing a corrective eyewear some of you may have an astigmatism and that cannot be fixed by just changing the diopter right. So, any problems of near sidedness far sidedness you can just play around with the diopter do some adding and subtracting and fix it however you like. And then in in a in a head mounted display if it has adjustments you could move the lens forward or backwards away from the screen to compensate for your near sidedness or far sidedness. However, you cannot compensate by just moving lenses back and forth for astigmatism you have to design some kind of corrective asymmetry into the lenses to fix this. So, there is some examples you may have seen before where some some letters may look sharp along one direction and then blurry along the other or vice versa and there is no way to your brain may try to find an intermediate focal length, your eyes may try to find an intermediate focal length to try to bring everything roughly into focus. So, astigmatism is the fourth one and the fifth one is called coma sometimes called chromatic aberration instead of chromatic aberration I will just call it coma that is derived from the word comet because it appears as a kind of comet image here. And this tends to happen when the when the part of the image you care about is very far away from the optical axis. So, this is a central optical axis here and these rays are coming in at an angle we are still looking at parallel rays for this and so at the focal plane you get these kinds of patterns that emerge. This gets particularly worse for thicker lenses because the waves are getting offset and shifted as they travel through the lens and you get these kinds of this kind of repeating patterns. You might have if you have ever played with a magnifying glass on the sun you may have seen some kind of repeated patterns like this. You may also see it sometimes in photography or in movies you may see some what appears to be a bright spot, but then a bunch of smaller bright spots trailing off from it. So, that is an example of you know this this coma pattern you get it is an example of what is called an airy pattern a i r y pattern. This will also happen in diffraction. So, you may have seen very simple examples in physics classes where you form a slit in a material and then look at how the light diffracts as it goes through that and the wave fronts will tangle in some kind of complicated interference pattern and it will generate stripes. So, that is the kind of thing that is going on here that is an example of an airy pattern. This coma is an example of an airy pattern and you will indeed see these kinds of things if there is a very bright distinctive say one pixel is lit up and it is over at the periphery it may appear as some kind of comet pattern. So, so it is harder to find those, but I have seen it happen before. So, that is all of the optical aberrations that I want to cover. Any questions about that?