 Which brings me to the next topic which again this is still under perception of motion and it is stroboscopic or scopic apparent motion. So, again let us think about frames per second FPS now it would be very nice to just talk about frames per second as if I am just showing you one image and then a second image and then another image and it just quickly flips through like that is how you could do it if you draw images perhaps on paper and go flipping through them although of course it is not exactly right because the pages are flipping down and there is some partial obstructions in between. And the same kind of thing is true in digital systems the pixels do not quite switch at zero time you know they do not switch instantaneously all the pixels are not necessarily changing synchronously there are a lot of strange artifacts that are going on and I want to talk about how that interfaces with your vision system and brain in some way. Then kind of general thing to think about with regard to showing you a bunch of frames is there is pulsing versus being always on. So, draw a kind of picture of what I mean here this is the pulsing case versus. So, it may be the case that in the pulsing and if I were just flipping pictures to you then here is what I here is what this would mean inside of a very small interval do I have my other color soil inside of a very small interval. So, maybe this is 1 millisecond I just flash an image at you ok I just flashed an image for 1 millisecond and the rest of the time it goes blank. So, this entire time in here which could be as long as maybe 99 milliseconds up to 100 milliseconds or so the photoreceptors may remember let us say the signal. So, this is blank completely blank time. So, if I just give you quick pulses like about a millisecond that is enough to charge up let us say your photoreceptors. So, that an image will be perceived and if I turn it off again as long as this image is bright enough over that short period of time it will be a complete picture will be stored let us say if I do this set of pixel by pixel basis or I draw an entire continuous image to you and flash it to you very fast for a millisecond that is enough for you to receive the stimulus. So, one possibility is to do that the other possibility is each one of these time periods here I am trying to show you some kind of frame rate let us say right maybe it is 10 frames a second in which case the time from pulse to pulse would be 100 milliseconds. So, every 100 milliseconds here I might just change the picture right. So, one possibility is to show you a picture for 100 milliseconds and an instantaneously switch to another that is always on. The other one as I said is I can just pulse and unblank. I am showing a just pulse and unblank case because our brains our vision system is capable of handling this and we have in fact, been doing this for a very long time and one of the motivations was for cathode ray tube displaced old television displays. They ended up pulsing the images on for a very short amount of time based on the amount of time that the phosphor remained on the the elements all across the screen remain on and they were dark most of the rest of the time, but your brain does not perceive the dark times because it is capturing enough photons from the pulse ok. So, that is one thing to keep in mind and now I want to talk about various frame rates give a kind of historical perspective as well. So, let me write over here um how many frames per second frames per second do we need and it depends on what you want. So, in terms of hertz if we go as low as 2 hertz we end up with the beginning of perceived motion. In other words if I just show you 2 different images I have a maybe a ball it is in one location in one image another location another and I just oscillate back and forth at 2 hertz. That is already fast enough so that it will appear to you like the ball is moving back and forth. It might not be a very smooth or continuous motion, but you will at least perceive some motion in that um. If you get up to about 10 hertz then um you no longer perceive frames individually. One of the most famous videos of this this is considered by many to be the oldest movie. This is by a Edward Muybridge is the horse video from 1878. So, there are I believe this was captured at 25 frames a second, but they could play it back at various speeds. And they they found that people cannot perceive individual images anymore once it is up to about 10 frames a second. They actually had a horse running across a field with a bunch of cameras a sequence of cameras set up in the field and the horse triggered the cameras by tripwire so that they could all take pictures and they could assemble this. Rumor has it there was a bet about whether or not all four of the horses feet would be off the ground at any time which is a very hard thing to see without a high speed camera. And I think there is a frame and there were all the feet are off the ground. So, this looks like motion right um I am not sure how many frames a second this particular one is I believe this is about 10 frames a second right now that we are looking at. So, it is hard to perceive individual frames, but um it is right on the threshold. So, you may be able to. So, if you go up a little higher um 16 you have early silent films right old silent movies um or old home movies right say in the 1960s and 70s people are buying video cameras and making home movies with and showing them on projectors and in that case about 16 frames a second um video ah let us see here. What is interesting about these old time films is that this is musty suffer from 1916 um it is about 16 frames a second, but what is interesting is they hand tune the rate of playing the frames. So, it seems is it uncomfortable to us does it make us nauseated um I would say no, but you can tell of course, that the motions are not realistic or captured exactly right in terms of what they would be in the real world, but it works somehow and in fact, this is a form of comedy at that time and it is perfectly acceptable to us. So, I guess one of the reasons why I want to show you this is that there is perception of motion the frame rate is actually variable here and was hand tuned. This is before they matched audio to the video, so they could do anything they wanted to it then they did not have to worry about the audio getting out of sync and it gave enormous opportunity to make entertainment. So, I think virtual reality is in a very similar place in that you do not have to get all of the aspects perfect you should just make sure people are not getting sick and if there is something you do to distort the video in some way or whatever it is you are presenting in some way if it is more entertaining or or more appropriate for the application you are trying to solve go ahead and do it. So, there is a lot of room for exploration here you do not have to make everything completely realistic. So, I just thought it was very interesting based on the technology of the time and the fact that there was no audio track they had the liberty to vary the frame rate and nobody seemed to complain it was all just fine. I thought that was great. Eventually the motion picture industry standard got up to 24 frames a second that is where it mainly is today. At 24 frames a second if you show these frames on a standard projector at 24 frames a second you will see flicker right. If I just have these frames pulsing on and then going black for the rest of the time or even in this case if the frame is on and then quickly changes to another you will perceive some kind of flickering. So, people started making multi-blade projectors that will show the same frame multiple times. So, you can then turn 24 into 48 with a two-blade projector to avoid or let us say reduce flicker. Really it is reducing perceived flicker I guess technically there is twice as much flickering happening, but it is reducing perceived flicker. Also 48 is a new standard that is been proposed for movies James Cameron other movie producers are pushing for higher frame rates there is a backlash between some movie fans and technologists as to whether it is whether they really want to have that level of temporal resolution at the movies or is it just you know do they prefer the way it is always been for very fast moving objects that come very close to your field of view. They look quite different at 48 frames a second if you are actually drawing 48 different frames rather than just using a two-blade projector. So, people are debating that right now whether we really need 48 frames a second. If you get up to 60 Hertz that is the most common rate for PCs and smartphones. I guess I skipped over classical television here. What was PAL at? PAL television 25 frames a second I think right was it not and NTSC in the U.S. we used NTSC which is 30 Hertz. So, I think it is 25 was PAL because I see Professor Monty is nodding right some of the older folks know right. So, that is very close. So, just more than it needed to be for the motion picture industry standard. At some point when they started making big computer monitors and people started sitting close to them there became a greater perception of flicker again and they had to increase the frame rates even more. This is very interesting because it is related to what is happening now in virtual reality. So, at some point when we had big cathode ray tube monitors people were sitting close to them the flicker they perceive at the periphery ended up causing a problem. So, it became the case that at 60 Hertz they had noticeable flicker 72 Hertz it was the minimum ergonomic recommendation. Personally I still would get some kind of headaches from 72 Hertz. So, there may be thresholds one at which you can see the flicker definitely see it there may be another threshold where you do not directly perceive it, but it may still have some long term effects. So, if you are sitting in front of it for 20 minutes you may get a headache or fatigued more. So, that is something very careful and important to pay attention to here this happens over and over again in virtual reality and over and over in human perception. There are different levels one of them is where you are consciously aware of the problem and another level may be where you think you have fixed it, but it causes at some subconscious level some trouble later on it may fatigue you. And I talked about the VOR gain changing for example, and that may be something that is at a subconscious level is fixed and you are not aware of that, but it may cause you fatigue later on some other kind of discomfort. So, eventually if you get up to 85 Hertz this was deemed to be comfortable for everyone. And so very often monitor timings were up in the 80s for the refresh rate on CRT displays. Once we went to LCD monitors the display rates went back to the frame rates went back to 60 Hertz because a flicker was not a problem because the pixels are always on. So, then the flicker is not as much of an issue anymore there is not really a perceived flicker, but when we had flicker on CRT monitors the rates got pushed up like this. So, this continues onward on our quest we go down for higher and higher frame rates. So, I had 72 85 which are right there currently you have seen a lot of people targeting 90 for consumer virtual reality headsets. I would like to explain a little bit of why people are saying that and there is another threshold which I guess I will put 1000 here just to really complete it all. And this is where I can still perceive flicker there is a certain experiment you can do which is very interesting which is you know grab your household function generator wave form generator. So, you do not have one in your house probably, but go to an electrical engineering lab and generate a pulse wave at 1000 Hertz and then connect it up to an LED right put a resistor in there or something and then move it back and forth like this. So, fast that your eye can not track it because otherwise it will try to keep the same image on the retina as you move around. So, move it very fast like this and what you will see is a kind of zipper pattern appearing through space turn off the lights and just move a red LED back and forth for example. And if you do that even at 1000 Hertz you can perceive the flickering of the LED. And the reason why is because it is going to generate multiple images across your retina even moving at that fast speed because your eye can not keep up and track it. So, this implies I think that even if you had a very fast display going at hundreds of Hertz if you move a virtual object through it very fast you will still perceive it as being a flickering object right seems unavoidable unless you go perhaps well beyond 1000 Hertz. I think people can see it even at 2000 or 3000 Hertz before this effect stops. It all depends on how fast you are moving the object right it has to generate enough images across the retina. Has anyone seen that phenomenon before? The very simple experiment to do if you have an electrical engineering lab around you have very simple wave form generator and an LED circuit. So, just move it back and forth. So, that is something to think about is even in spite of this whole progression I can still design a scenario where if I have a very sharp bright spot let us say very small bright spot and I want to move it very quickly through VR even at a very high frame rate I may perceive some kind of flickering because my eyes cannot track that using smooth pursuit. But what is going on here why is it that 60 Hertz is not enough people are making a big deal out of 90 Hertz right now. The headsets that you have in the lab are at 75 Hertz and that is interesting because that is very close to the minimum ergonomic recommendations that people had for CRT monitors in the 1990s. And 90 Hertz is very close to the comfort for all let us say thresholds for CRT monitors from the 90s. So, these things are very closely related and I will explain why in just a bit. Questions?