 Hello everyone, welcome to Computer Science E1. This is lecture 5 Multimedia and so I know everybody is bleary-eyed from their days on the beach after spring break And so I thought that to begin we would start with a fun animation Representing multimedia. Oh, I'm gonna have to plug this in. Let's see here Now this would be very easy to just sort of look around on the internet and find funny things and show them for the entire Two hours, but this does actually have some relevance to some of the things that we need to talk about today And so in particular this animation that you saw Represents a certain file type or it's made from a certain file type called flash And so many of you have probably seen flash even if you don't know and you've probably heard of it Perhaps even if you don't know exactly what it is But flash is basically just a plug-in For browsers on the internet for browsers on your computer more specifically that allow you to Display various types of files and this is just one of them which includes animation And so traditionally flash was used for a lot of animation and a lot of what are called vector formats So something like that looks like this So what might look a little bit different as opposed to the previous animation that we had seen besides obviously the contents itself Is how it is actually drawn and in particular if I resize this page We can see that even though it resizes what happens is that the The lines that make up these horses do not actually become Pixelated and so this is yet another type of of a flash example And so basically what you do with this one is this is an example of interactive Multimedia where it not only includes sound and animation like we had seen before But also what we can do is now click on each of these horses to be able to get them to do something And you can witness my terrible timing as I try to get everybody Aligned properly and eventually what you get is this very nice. I know it's horribly off But that's the the downside of live demos But what's more interesting is as we scale this as we change the size of this particular window It doesn't actually impact the sharpness of the image that we're looking at and this is in Dark contrast to pretty much everything else that we see on the internets and so just this other just this other Flash animation for example what you saw was it was basically this You know this collection of palettes and windows in a Microsoft Word style or Adobe Photoshop style application And you could see the pixelation from each of these things So let me point out now if we restart it from a specific point look at the left side How you can start to see? Individual pixels and what a pixel is and I can zoom in even more to make this even more apparent What an individual pixel is is exactly this it is just a small square Representing Some color that you can see on your screen in fact your screen is made up of thousands of these small pixels And so we use the combination of pixels not only to make the elements that we see on screen But we also change them rapidly like this so that it forms this illusion of animation or motion Like what we can see so there's this important distinction then between different types of files One of them is this so-called bitmap where when we look at an image and it's made up of individual pixels There's a limit to how much we can zoom in on that image before we start seeing the individual pixels And this is compared to let's see. I don't know if there's a way I can pause this Compare this to a vector type graphic where you can then use any resolution and it still looks just as sharp So if we were to zoom in on this as much as we possibly could what we would start to notice Let's see. I don't know if this is nope It's not working, but um if I re as I resize this window as we saw it doesn't matter what size it is It still appears to be sharp. There's no noticeable Pixelation in the horses. So this again is two distinct types of of files two distinct ways that we can represent Graphics one of them is this vector format like we can see here And the other one is this bitmap format and I mentioned before that there was this plugin called flash and flash does a whole lot of things But it does actually include capabilities to be able to display vector formats like this and animate vector vector Drawings, but also it can display bitmap images So when you go to youtube for example and you try to display a video by default it loads a flash player And that shows you a bitmap image It's just a movie that is made up of multiple bitmap images And so you work with this is not Although this terminology might be new these types of files are not new In fact, you look at you work with bitmap files almost every day And so when I say bitmap realize that there is actually A file itself that is called a bitmap type and yes, that is a type of bitmap file But what I'm talking about is more general this actually applies to pretty much anything that just makes up Through the use of rows and columns of pixels the end results is just an image And so this is in effect a bitmap image and when we zoom in on this image We can see that it is in fact bitmapped because we can start to see pixelation We can start to see individual pixels that make up this particular file And so this is an important distinction between bitmap and vector and most of the time When you're working with images in fact almost all of the time you are working with bitmaps If you're taking a photo with your digital camera, for example That's taking a bitmap photo even if it's in a different file type than than bitmap If you're working if you're looking at a video online, for example, most likely is it going to be The individual frames are going to be bitmap images just because the Combination of them is is sort of the easiest for us to do So the difference between each of these is that bitmap is basically it's what it sounds like It's just a map of bits that make up all of the pixels that we see in an image So it just is this a rectangle full of pixels with different colors And there's a maximum limit to how much we can actually zoom in on this bitmap before we start seeing The individual pixels now what's different about vector formats is that those actually use mathematical Equations in the format to describe how this line should behave So it says okay This is actually going to be a straight line going from this point to this point and because it's mathematical Then they can just recompute The what the line should look like depending on the resolution itself And so there's this resolution dependence versus resolution independence There is this idea of resolution dependence in a bitmap where it depends on the resolution that you Were looking at it to see if it's going to be a sharp image or not Whereas and compare this again to vector images where this is resolution independent because it doesn't matter How high of a resolution you make that image it is still going to be Sufficiently high that it's going to be sharp just because the mathematical equations then describe how this This particular image should work and so it's very very it's well It's much easier for us when we're thinking of digital Cameras and all of these things that actually make images It's much easier to think of them in terms of bitmaps Just because it's much easier for us to define individual pixels the color at each individual pixel And this makes a lot more sense to us rather than using some vector formats And so though it is definitely possible if you use something like adobe in design for example You can actually create some vector graphics so that they can scale up to be absolutely humongous without any loss in quality Most of the time that's sort of that's sort of Kept to things like short animations like the horse one that we just saw and And also the things like logos where resolution independence is actually very important so that we can scale the size of a logo very independently Bitmaps it's although it's very easy for us to do we do have this This limitation of the resolution and in fact you're familiar with these sort of connotations of resolution that a lot of digital cameras actually use like megapixels for example that actually describes How many pixels an individual image might have so if a camera says that it has eight megapixels for example That's just saying that The number of pixels in width times the number of pixels in height That's the total number of pixels in this entire image So if you have eight megapixels then most likely you have a camera that's something like 3200 pixels across by 2300 pixels down or something like that and that actually describes then the size of the image And if you try you can zoom into that image and blow it up to will be a hundred percent And if you are looking at an image that is 23, let's see, uh, 3200 pixels across by 2300 pixels down on your own monitor Then most likely that image is going to be larger than your monitor. So we're talking about pixels so keep in mind that pixels sort of Are this universal way of describing a dot on a computer screen And so we can describe not only a pixel in an image in that An image is made up of multiple pixels, but also Projectors or screens are actually made up of pixels as well In fact, we can see the resolution that the current screen is operating at and that's at 1024 by 768 So what this is telling us is that there are a certain number of pixels on this projector display There's 1024 across and there's 768 down So if I am looking at this image this eight megapixel image That was something like 3200 pixels across by 2300 pixels down And I look at it at full size on this display. How is that image going to appear? So i'm just displaying this massive image on the screen and what is How might this look to us? So yes, we will see it's completely zoomed in at 100% So each individual pixel of our screen matches the an individual pixel on the image itself But there's something funny here. There's a difference in the number of pixels. So what does this mean? What's what is our computer going to display if we try to do this? Any ideas? So it's not going to compress it. It's just going to you're just going to see However much will fit on the screen. So in essence, you're just going to see a small fraction Of the actual image And so there's this idea of scaling. So what we what we tend to do sort of By default in in when we're working with computers is that we like images to sort of be as large as possible So what we might do is we might resize the image to be smaller on the screen And so what that means is that we are taking this absolutely huge image Say 2300 pixels or no rather 3200 pixels across by 2300 pixels down And we're resizing it. We're making a little bit smaller so that it now can fit Within the resolution of our screen and this is an important thing to realize that there are in fact Pixels and we can call pixels the same thing across multiple formats But there's this additional level of scaling that tends to occur Whether or not we are zooming in on something to make it appear bigger or we're zooming out So to speak to make it appear smaller and to make it fit Within our screen. So zooming out on an image It's not really going to impact our resolution dependence all that much right because if we're zooming out Then the pixels in an image are perhaps a little bit smaller And so in in essence we have enough pixels to display everything that we want to so in essence This image here was taken with an 8 megapixel camera But this is the entirety of the image and so in order to fit it on this screen This is just like what we had talked about it had to be resized down It had to be scaled down to fit the screen Now there's nothing particularly complicated about this concept We're just sort of trying to formalize some of this terminology into Scaling and resolution dependence and these sorts of things and it does actually build To to some important things that we'll get to in just a moment But right now I just want to mention all of these things because it is in fact important Now if we go the opposite direction, let's say that we zoom in on an image So that it's at 100% and what that means is that one pixel in the image matches one pixel on our screen And that's sort of the ideal Resolution in other words that's is that's the highest resolution that we can get Before we will start getting blockiness or pixelation because as we zoom in more That same pixel on an image has to become bigger and bigger and bigger So that we can continue zooming in and so this is again what we're seeing here We're just zooming in on an image so much that we are just seeing the individual pixels that make up that image Yes So if we have an image that's enormous and we're trying to fit it on the screen does that entail Compression so yes and no so If you just do If you're just looking at an image and you zoom out on that image Then perhaps what's going to happen is that it's just going to resize that image and no Recompression the image itself won't be modified, but if you're actually modifying the image Let's say you open up this image in some in some software like iphoto or photoshop or some other image manipulation software And you actually tell the software to resize the image then it's actually going to do that It's actually going to resize the image so that it becomes smaller And it's going to then cause the file itself to be smaller as well Just because there's there's fewer pixels in the end result And that sort of thing can actually cause a recompression when you're actually saving the image But we'll talk more so it's There's it's it is in fact independent of resizing the image But as a result of resizing and then saving the image can we be compressing it So strictly speaking I would say that no a compression is not necessarily making it smaller compression is talking about something a little bit different That deals with not the size of the image on the screen like this But rather the size of the image on disk how many bytes does it actually take to store this image on a disc? That's usually what compression refers to we will actually talk about compression and the different types of compression and just In just a little bit Okay, so we have This image we can blow it up to some enormous quantity like this And I have to say that um There's a lot of tv shows and movies out there Whose entire premise is based on the fact that you can zoom in on an image like this And enhance it so that you can see all of the details and frankly that is just total total crap Every time I see that it bothers me to no end whenever they zoom in on some image like this And they say enhance and then they can see the reflection of somebody's face in this little thing There's no way that you're going to be able to do that because this is the entirety of the data That this file has what you see here. That's it. You're going to be able to guess as well as a computer What's the higher resolution version of this? Dot whatever this is Is actually going to be and in fact if we see where this is coming from we can see that there's a this circle is Pointing out the small circle up here, which is basically just an imperfection in the wall So it's really it's really just like a bump or an indentation of some kind that exists in the wall itself And so this is not going to provide any useful details But the concept is the same. It doesn't matter if you're taking some really low resolution photo of somebody's Oh my gosh, why am I blinking on this? on somebody's License plate, that's the word so it's not like you're taking some really low resolution Photo of somebody's license plate and then realizing. Oh gosh. Okay. Well now I can just enhance it That's just not going to work and there's been Quite a bit of research that says that okay, this might actually be possible But realize that that also is not it's not exactly what you would think of there's been some recent Research that came out a couple years ago that said something like well We can actually recreate an image based on just a few pixels at a time But really if you read the fine print of this it means it It said that it can recreate the image with high probability But if only you had random Pixels from various points within the image not if you have something like this some uniform collection of bitmap So in short this enhanced stuff It's just a bunch of baloney and it's not really worth anybody's time to realize that it's going to be Useful now there is this concept of sharpening which is actually a valid concept Which what it does is it takes an image that looks like this and it might be pixelated And it tries to make it look a little bit smoother tries to and it tries to Sharpen the edges a little bit just so that it makes it look a little bit more pleasant to our eyes Now that actually does exist that technology does exist, but it doesn't actually give you any additional information It's just a trick. It's just sort of an optical trick To get our eyes to make it appear as though it is a bit better or a bit smoother even though There's not actually anything that's um more enhanced about it. Yes Like Okay, right so Yeah, so in fact So i'm i'm less familiar with sound engineering than i am with with images, but The concept is pretty much the same and sound you have a finite amount of data you have However much your recording has that's how much data you have especially if it's digital if you have some digital file Some digital representation of that you're not going to be able to get any additional volume out of it Or rather any additional resolution out of it So if if it's some orchestra playing for example and you hear sort of a faint sound in the background You're probably you can probably adjust the volume to see if you can hear it better But you might not be able to fully understand what it is if it's very very faint and so when They're talking about cleaning up audio, especially old recordings and removing the pops and the hisses and that sort of thing Um as far as I know what they're doing and there's probably there's probably some sophistication involved that you know I'm not very familiar with but i'm sure what they're doing is They're looking at the sound and and figuring out what it's supposed to sound like because perhaps They know what the progression of notes might be and so they might be able to recreate that progression Of notes within within a sound and so it's it would be sort of similar I think the analogous situation to images might be if you um If you have an old image for example, and let's say Uh a corner is is is torn off or burned off or something like that and you want to recreate the original image You can go based on context and figure out what might be in that corner in a in a in an artist might be able to recreate What that image probably looked like but it's not going to be the original even though it looks very good and even though it's it's Uh it it's almost perfect. It's not quite the original if that if that sort of makes sense And so I think that might be the closest analogy to sound engineering in this in this respect Now um, so when we're talking about uh in okay, so we mentioned Enhanced for example and how we can try to blow something up as big as possible Uh, and we're not actually going to get any additional detail out of this, but let's say let's well Let's go back to this idea that we don't have this capability There's no way that we are going to know What the original detail was and this is certainly true But we can get another couple of interesting tidbits out of this image just from looking at This blown up portion of the image so first of all realize that we can see the individual pixels But that each pixel has its own color So somewhere there is some way of defining the amount of color or which color is actually going to appear In an individual pixel And this is very much like um You can think of this very much like what it is what it sounds like It's a bitmap when that what that means is that for each pixel There's a couple of bytes of information describing the color in each in each individual pixel So for each pixel that you see here, there's about three bytes of information Defining the color that makes up this information. And how do I know that? Well most of the time when we are talking about colors and images It comes in a variety of formats. It might come and say something like eight bits Might come in something like 16 bits or even 24 bits And what this means is the number of bits that are provided to each pixel To define the color within it So if we have eight bits of color information that means we have how many bytes per pixel Yeah one byte per pixel and so one byte represents how many distinct values We talked about this so if we have one byte then we can have how many different Right 256 discrete values So if we have an image that then has eight bit color for example Then what we are saying is that that image at each individual pixel we can only have one of 256 different colors And this is not going to be very much right? We will actually be able to look at an image Especially if it's if it's of a photograph which has maybe thousands or even millions of different colors within it We can look at a photograph that's made up of 256 images. We can realize that that's not enough It looks funny. It doesn't look like there's quite enough colors there But there is other bit depths available like this 24 bit bit depth for example This means that we have three bytes of information And so that's 256 values times 256 times 256 in essence We have I think 16.7 million different values available to us For each individual pixel for different colors. And so how is this broken down? We'll realize that when we have eight bits of information or rather 24 bits of information It's broken down into primary colors And it's not the same primary colors that you might have learned in elementary school But instead it is this common set of primary colors that Computers tend to use very frequently red green and blue rgb In fact, you might have seen rgb in use in a variety of contexts And it's true and pretty much anytime there is a computer or some Computing device that has to display colors. So tv's for example This applies to especially modern tv's that have pretty sophisticated electronics computers definitely use this terminology All sorts of stuff. So we have three components a red a green and a blue component And each one is allotted one byte of information So what this means is that for each of these three colors? We have 256 discrete values This may not sound like very much But remember putting all of this together if you multiply 256 times 256 times 256 you get millions Of different colors and in fact again, this is something that if we take a look at our monitor This is something that uh translates relatively well. So let's see. I'm not sure it's on this Screen if we go to color here, we can see it's not on that one. It's on this one. No, where is it? Oh, they must have removed it in this version. So, okay So my live demo is now a year old But typically when you look at the display options for a computer It actually tells you how many what the bit depth you actually want and an older In max older than this one it would actually display Thousands or millions and most of the time millions would be displayed and on windows computers Usually it tells you something like 8 bit versus 16 bits or thousands or millions or some similar thing And that's basically referring to the same thing. How many bytes are you allowing at each individual pixel to make up All of the individual colors that exist within an image. Yes What is the Hertz rating measuring? Yes, so uh a Hertz is basically something that occurs Every second so you might have also seen Hertz in the context of cpus because cpus will do some Something in say gigahertz and so billions of Hertz or millions of Hertz depending on the age of the cpu In this case, this is how often the screen is refreshed And so right now our eyes Our eyes have a minimum limit below which if something is is Uh refreshing at a lower limit than about I think it's 15 or 16 frames per second or so 15 or 16 times every second Then we will actually be able to notice that there is some refreshing going on It won't appear smooth the motion won't appear smooth to our eyes So generally the sweet spot is something like 60 Hertz or so just because that's refreshing fast enough For our eyes to make it appear to or for it to appear to our eyes that it's this smooth motion so um Just like movies and uh pretty much anything where there's Uh a recorded motion going on what we're seeing is not in fact, you know very smooth motion It is in fact made up of perhaps Dozens of individual frames in each uh in each second So that's basically all that this refers to here So you'll notice that we can in in fact increase the Hertz But it's you're probably not going to notice In this case just because there's not motion occurring on the screen And so there's really no reason for you to uh to notice much of a difference and in the Hertz in this in this case Okay, so coming back to this so now We have this image and I mentioned before that we have an image that's um About 3500 by 2300 pixels in size and so in grand total it's about eight megapixels, right? So eight million pixels in total made up this particular image But now i'm telling you that in addition to that for every individual pixel We actually have three bytes of information for all of the colors that we can represent So what this means is that for every pixel there's three bytes So if we have eight million pixels in this image Times three bytes. How big do we expect this image to be on our computer when we're actually writing it to the hard drive eight million times three bytes is 24 million bytes, right? So if we we can do some division and realize that that's more or less about 24 megabytes Right, and so that's generally what a totally uncompressed 24-bit color image is going to be on our computer It's going to be 24 megabytes and that's pretty big Of a that's a pretty big file for something that's Relatively simple at least in terms of modern-day multimedia where we have Sounds and images and motion and all of this stuff. So obviously like we'd mentioned before There's some compression going on to be able to make this image quite a bit smaller and in fact Every time that you take a photo with your digital camera Your camera is actually performing some compression for you so that it actually will write a much smaller version of this file It's it's compressed. It's not resized. This is again as goes back to this Concept that we were talking about before it's not that it's necessarily resized. It's that just it's using fewer bytes On the on the digital camera or on the camera's memory more specifically So if we have now a digital camera so realize that a digital camera is basically made up of a sensor That's included in the back and there's a bunch of mirrors that that a digital camera might have and in fact This is a Sort of a fancier SLR camera So it's not something that you would have say in your camera phone or even a small point and shoot But this is sort of the type of camera that people tend to associate with Professionals or advanced damageers that sort of thing that you would go out and spend Quite a bit of money to purchase for for one of these SLRs But once you get past all of the all of this stuff that's in this camera It is in essence just a sensor in the back that is made up of individual pixels That read the light that that scene has and so this light is then captured by this sensor And converted into this digital format where it is then saved in this sort of bitmap type file So I mentioned before that bitmap is a type of file It's not necessarily the file type and i'll talk more about the distinction in just a few moments But realize that when we take the photo This camera then converts all of those individual pixels To some gradient of color based on this same idea where red green and blue is broken down into Or each of the colors is broken down into individual Components involving red green and blue and then saved in its memory. And so there's actually um It's Digital cameras actually a pretty sophisticated device and it's in fact quite a bit more complicated than this and in fact The digital camera sensor itself cannot actually see color In essence, so it is all digital cameras Can see black and white and so you might be saying okay Well that sounds like the stupidest thing I've ever heard because obviously I can take a picture and it's in color We'll realize that um the designers the first engineers of this realized that they could do a trick To so that they can make it appear as though a digital camera can see color In essence what this means is that for each pixel in a sensor There's actually a filter on top of it a red a green or a blue filter that's on top of that pixel So that only that color will pass through to that pixel So that even though that pixel is then only able to determine light from dark It's then able to know based on its placement underneath this filter Which is called a color filter ray just as a sort of the terminology for this particular type of filter So this color filter array then filters the color Into brightness that each individual pixel will then be able to Monitor then we'll know what the brightness level is and then in the end result is giving you a nice color image Using these breakdown values of red green and blue and so this is not something It's so low level enough in the digital camera that it's really not anything you're going to be able to notice Even when you use the the silly black and white modes in digital cameras It doesn't even remove the color filter array. In fact, it still uses it and then converts it to black and white later There's a whole bunch of really interesting things. I think and The we actually go into quite a bit of detail. There's a another class So if I can feel let me just blab a little bit about it There's another class computer science e7 if you're interested in digital photography That's the way to go we go into a lot of detail about Cameras and how they work and why mega pixels is not the end all be all monitor of a Of performance in a digital camera so on and so forth might be something that you are interested in So a digital camera then when it takes a picture can basically record Essentially the most common are two distinct file types And so most of the time it will save a file and say the jpeg file format But there's also another type that's much more ambiguous called the raw file type and the raw file type will actually save All of the all of the raw bytes all of the individual raw bytes that make up the image And so when we were talking before about how an eight megapixel image might be something like 24 megabytes it could actually in fact be quite large because there's not much compression That happens in a raw file now. This is a little bit of a lie It's just it just that it happens to be a different type of compression But I'll point out the difference again when we start talking about compression in just a moment Whereas jpeg is the much more common version of a file type that a digital camera will write and that is pretty heavily compressed It actually will take an eight megapixel image bring it down to something like one or two megabytes in size Which is just a small fraction of what the original data Actually was but because the compression is relatively smart. It doesn't actually appear to us That there's any loss in quality even though there is in the background They had to throw away Some of this data Now to compare this once again to bring this full circle to what we were talking about early on We mentioned before about how there is this concept of vector graphics about how there's these Graphics that even though they look like they're made up Individual pixels and in fact they are when they're displayed on the screen They do in fact have to be rendered They do actually have to be displayed as individual pixels But as we zoom in on these in on a small portion of a vector graphic The computer is then able to recompute What it should look like based on how far zoomed in you are and again because it's just made up of of mathematical equations Really doesn't matter how far zoomed in you are you can just see that it looks sharp And in fact you can see these sorts of jagged edges perhaps where somebody was trying to draw sort of a smooth sort of a smooth edge here But in fact was just had a just had a little bit of slop on on either end And this isn't necessarily a problem with the the vector graphic itself It's just the way that it was designed It was just designed to sort of look like this sort of cartoonish and on the right you can see This same image before all of the colors are put in so just we have this These mathematical equations that define this outline for this image And then we can use more math to say okay Well all of this color that looks like this should be in here for example And just display the color within some geometric shape and display that within Within this image and so if we have an image and so we've talked about how if we have resolution So if we have an image that is in fact A bitmap and what we want to do is actually Blow it up so that it's a very large size Generally what we have to do is have a very high resolution image So what we're saying is that if we have an image that is going to be a bitmap file type like this And we want it to be absolutely massively huge and still appear sharp without any noticeable pixelation Or or any sort of problems like that then we have to have a very high resolution within this image And so this is where The sort of idea of having high megapixels is really has really sort of taken off and that okay I have way more megapixels now on my camera So now I can really blow up my image to be absolutely massive and I will have great resolution out of it And yes, that is true to a degree But also realize that there's a lot of additional complexity in a digital camera that can define Whether or not the the number of megapixels that you have in that camera actually of high quality As opposed to so quality does in fact matter for these individual pixels But if we have to create something like a very big billboard for example, and this is going to be Dozens of feet wide perhaps and dozens of feet tall Or if it's just the standard billboard and it's dozens of feet Wide and perhaps just a dozen or so feet tall Then ideally we would need to have an image that is very very high resolution as well And so we might say then that using something like a vector graphic would be ideal for this And in fact, this is why one of the reasons why it's very ideal to make Logos or other other designs that are meant to scale to be very very large In vector graphics just because then you don't have to worry about pixelation You don't have to worry about blackiness as you zoom in on an image or as you blow up this image to make it much much larger Now the it's sort of a white lie saying that we have to do this for billboards because There is another factor in resolution as well And that is how far away you are from the object that you're viewing if you're very very far away from it Then we can only see you know, it's it's as if we have pixels in our eyes as well And they're only you know High enough resolution to be able to capture a certain number of things so billboards actually Though I use that as an example. It's not strictly So that it has to be such high resolution just because generally when you're viewing a billboard you are Many many feet away and so when you actually approach a billboard It can actually be made up of pixels that you can actually see once you're close But this is sort of analogous to you taking your screen for example And looking really close at it so that you can see the individual pixels that make up that screen It's the same sort of concept You know for a billboard here So like we said before just to bring this idea Back to the screen this this projector itself. This is made up of only a thousand pixels across and in fact standing right here It's very easy for me to see the individual pixels that make up each of the well that make up the entirety of this image But perhaps sitting where you are now. It's not quite as easy to see those pixels It's the same sort of concept where Not only do we have to take resolution into account But also the distance the viewing distance away from that image But in terms of computers every time we're we're talking about resolution It is an absolute value. So we are saying that okay This image here is made up of a thousand pixels across and 768 pixels up And that it makes up our image here Now when we start talking about Creating images there are of course other things as well So just beyond images and animations there is another type of multimedia as well Whereas multimedia is just it's what it sounds like it is just multi Types of media whereas it's probably some combination of images and graphics or sound or it could be any one of these things or it could be Of movies or it could even be something like games and games, especially 3d games Take the sort of concepts that we've been talking about to a different level So because they're designed to be to have this sort of 3d perspective to them Generally what they do they do a couple of things in order to make them the way that they are So the very first thing is they make a wire frame out of the object So this wire frame actually defines the edges of this object in a 3d space very much like we might use Vector graphics for example, and so this uses some math But more but more specifically wire frames tend to use Very very simple polygons like perhaps rectangles or triangles most of the time And so you can see in this wire frame that it's just made up of a whole collection of triangles and rectangles That's in in combination make up this what will eventually be a face And this again is just defined with some math within the computer And so in order to give some texture in order to give this face Some some appearance besides just a very blocky very computer looking wire frame They do a couple of very interesting things first. They fill in each of these Polygons with some colors like you can see here, and it's a very basic It's definitely an enhancement from our initial wire frame, but it still looks sort of silly Then they apply some additional math to do some nice shading and to smooth out the polygons a little bit to make it look like The face is actually smooth as opposed to just the collection of polygons and triangles and And rectangles and then finally they actually apply a texture to the to this image And the texture is in fact a bitmap And so they just take this bitmap and they just sort of wrap it around This object that exists this mathematical object that exists in memory And the final result is what we see and so if you ever play a 3d game for example And you sort of you're playing a character and you go up really close to a wall and you look to see what that wall looks like You can sort of see that that wall is made up of individual pixels Just because the texture that makes up that wall is in fact a bitmap whereas the All of the shapes within the world itself tend to be more wire frame And and therefore are made using more mathematical models And so those tend to be a bit better defined the shapes tend to be a bit better defined Than the textures themselves And so this is one of the big distinctions This sort of defines the way that a number of video games actually look just because of the way That they that they are designed and so we can actually take this concept of a wire frame to its To an interesting if not very illogical conclusion and do something like create An actual object out of a wire frame So this is actually a real picture that exists of somebody that had made A car out of a bunch of wire frames just to sort of give it the same Look as we had seen Just a moment ago and they're sort of from what I understand about this The story is that they had placed the image on the street and because it wasn't It was considered to be a car But it didn't have the necessary parking permits would actually got ticketed or something silly like that So it's sort of a funny end story for this non car that looks almost like it's a video game Just in an in an actual picture even though it is an actual real life wire frame represented by this car Okay So we've been talking about these different ways that we can create Images on a computer screen. We might use a bitmap image, which is sort of the simplest way It's just a collection of pixels Defined with a bunch of colors that we can then use to create an image or it can be a vector format Which uses a lot of math to define shapes and we can zoom in very far into that image because it's we it doesn't require Any particular resolution or we can use a combination of both Like they do in games to create all of this But we mentioned before how in order for us to be able to store a great amount of these images We generally have to compress them to some smaller state And so there are a variety of compression techniques But one of the simplest is something known as run length encoding And so one type of file called a gif Gif file actually employs this run length encoding and it's actually a pretty important Type of of compression that we can use to greatly save the amount of data in in image So we have here Images of two flags. They are on the computer Excuse me 300 pixels across by 200 pixels down And we can see that they're made up of three colors But because they're in a gif file format that each pixel can actually be represented by a byte or by eight bits So just 256 different values for each of these So if we if we assume that for a moment, how big should these images be on disk? So 300 pixels across by 200 pixels down is how many? How many total pixels? 60,000 right so 60,000 pixels And if we assume one byte for every pixel Then how big then should this image be more or less when we're actually looking at it on our computer? So one byte for every 60,000 For every of the 60,000 pixels we would get about 60,000 bytes right so just one byte per pixel. That's all that we have So we have 60,000 bytes or about 60 kilobytes for each of these images right and that's not a lot But also these images aren't very complex. They're not very large. They're pretty simple images But let's see if we can do a better job than that So the whole concept of run length encoding is to look at each individual row of pixels So it looks at the very top row in each of these images And it looks at the color at the farthest left column of that row So in the in the example of the german flag on the left here We see that we have a black pixel at the very farthest left farthest top row And then we say okay I'm just going to say In in in a quantity how many times this same pixel appears in this row And so it's essentially compressing it based on the length of the row So we can see here that the entire row is made up of black pixels So this really compresses down very easily to just basically a couple of bytes right the first pixel is a black pixel And then we can say okay repeat this pixel 300 times And compare that to actually having a black pixel followed by a black pixel followed by a black pixel Which is fine as we can certainly do that obviously that's how we display this But it's just not very efficient when we're saving it on disk So using run length encoding then we can store all of this stuff down into just a couple of bytes Of information for each of these rows So we can do the same for the next row which is again black the next row which is again black And then finally when we get down about a third of the way does it change Then we can store the same thing for each of these other rows of color as well Now how about the flag on the right? So we're going to use a similar method But is it going to compress as well the same or better? As the one on the left Based on this encoding technique this compression technique that we had just talked about Will it take more bytes? And I will give you a hint the colors actually make no difference whatsoever. They could be the same colors In both flags, but they are actually going to be different, which might be better Right the one right so the german flag the one on the left is in fact going to be smaller Just because we don't have to change the instructions quite so often So with the french flag on the right we can say okay the very top the very top row very far in this left pixel Is blue okay repeat that a hundred times, but wait a minute now. There's another pixel in this row. It's white Okay, so use that repeat that a hundred times now There's another pixel and it's red use that to repeat it a hundred times. That's fine That's definitely much better than the original uncompressed version But it does in fact use more data than the one on the left. Yes So My understanding is that each time it Each time it finds a new color it restarts and begins again So if we had to repeat some in color So if this wasn't actually a flag and it was just stripes like red white blue red white blue So on and so forth that were maybe a couple of pixels or even just one pixel across Then it would have to reset each individual time and it'd say okay First one is red for a couple of pixels then white for a couple of pixels then blue for a couple a couple of pixels Or rather the other way around blue blue white and red for a couple of pixels each and then repeat that same pattern So Yes, in fact, there is a very good point that there is a very good reason when This particular encoding does not serve us very well that is in fact for any image That's very much like a photograph where each individual pixel is perhaps very Similar, but not exactly the same in color as its neighbor So we can take a look at this image that is of an apple that has a uniform blue background For example, and there's only so much that we can do to compress this image in this same way Because the blue background is uniform and so we can very we can say okay The the the top row is all blue Is all blue pixels the next row is all blue pixels so on and so forth But we start to get a problem when we approach the apple itself because each individual pixel might actually be changed ever So slightly from the adjacent one and so when we actually have raw images or photos Then this same idea doesn't actually work quite as well And in fact this type of compression Is in fact a type of compression that's known as lossless All of the data that we're saving the original data that was used to create these these images can actually be recreated From the compressed counterpart What that means is that by looking at the by looking at the compressed data We can then reformat in its entirety all of the original data that made up these images, right? Just because we know the for the german flag There's 300 pixels of black in the top row then 300 pixels of black on the second row and so on and so forth And we can recreate that very very easily in its original uncompressed format Now this is in fact in comparison to loss or to lossy compression Where it works perhaps a little bit better for a number of images, but we do In fact then uh, let's see It does work better for a number of images, but we're not able to recreate the original image itself But before I talk more about that which is actually a different type of compression. I think I saw a question Did you have a question about uh? Oh, so instead of yeah, so instead of having um, so whereas jiff compresses very well for horizontal bands of color couldn't we have a different type of file that Compresses it vertically. So yes, we could but um, that is sort of an expensive task To compute both compressions and figure out which one might be the best Unless the the person that was encoding it or that was compressing it Knew the difference between the two and was able to then make a distinction that just add some additional complexity We certainly we certainly could do that, but just add some additional complexity and for whatever reason they decided to go with Horizontal run length encoding rather than vertical encoding So all of this this is how a jiff compresses its data But we've already talked about how a jiff also only has one byte Of color for every pixel and so a jiff might not be the best choice then for something like a photograph Just because there's very subtle changes in color And even though the colors might appear to be the same they are in fact ever so slightly different So as soon as we start getting to images that look like this Where there's much more complexity within it There's a lot of different colors within an image and we actually have to compress it In an entirely different way we'll realize that if we use the same run length encoding for this We're probably not going to save very many bytes at all In fact, the only bytes that we might save are going to be in these sort of overexposed areas of the image Where over here where it's like obviously very very white in this little patch of of water right here But other than that there's not going to be a lot of of capability for us to compress this So how then are we able to compress an image in actual photograph? In a in a much better way without sort of losing too much data Well in order for us to talk about that we'll have to take a five minute break when we come back We'll continue talking about image compression Hello everyone welcome back so the break before the break We were talking about compression of various types and how jiff uses run length encoding, which is a lossless compression type so that we can recover all of the data Original to the the image But if we want to do something like actually encode an image an actual photograph Then typically we have to employ some more advanced compression techniques and so Going into the how say a jpeg actually works is actually it's it's actually pretty complicated But basically they Divide an image into a variety of blocks like you can see here in a very badly compressed jpeg image here of the same crop And it will analyze each of these blocks and try to simplify it in some way So if you actually look at some of these blocks long enough You'll actually notice that some blocks are missing color for example, and so the block that is right Right here directly to the left of the bill of the hat This is in fact a block that's where they've just thrown out the color because the compression The compressor itself has detected that most likely People aren't going to see color in this region quite as well as they might see color in other regions And so there's some assumptions that go into compressors like like the jpeg compressor a lossy compressor Where it will actually throw away data that it seems unnecessary to the image And most of the time this data is imperceptible when we throw it away It's actually doesn't matter so much when we're actually looking at this image But throwing away even small amounts of data in specific regions can actually add up to huge savings In the overall jpeg image so we can actually remove even at a relatively high quality We can compress a bitmap image an original bitmap image down to just a few megabytes in size when it was originally Perhaps a couple of dozen megabytes But this was in fact a lossy compression what that means is that we cannot in fact retrieve the original version Of that every time we save an image As a jpeg file we are in fact lossy compressing it just a little bit And so it is throwing away some bits of data And so it's generally not a very good idea if you're making lots of modifications to a jpeg file It's generally not a good idea To modify a jpeg file Save it and then reopen it and then modify it a little bit more and then save it and then reopen it because eventually You're going to start losing more and more bits Away from from that image. Yes Right the difference between the two is that one the one on the left is basically an uncompressed Or it's a losslessly compressed image so we can see all of the individual pixels as it actually was taken But on the right this used the lossy compression But again, it's over exaggerated just so that you can see the existence of the compression Because jpeg is actually pretty good and there's a reason that it's One of the most popular image file types around because it does compress very well And the data that it does throw away generally we don't miss very much But it is in fact lossy and what this means is that if you overdo it Like we did on this right image here Then you will in fact pay the price and quality in your image So you don't want to go overboard with the compression because you can in fact get what are called lossy artifacts You can you can see artifacts as a result of the compression There's the the spray of water has become kind of smudged and we've lost color In certain regions and there's we've lost definition and others And there's just a variety of things that actually change the way this image actually appears even though And again, this is an over over an over exaggeration So had I used a lossy compression but not overdone it quite so much Then perhaps it would look a little bit better or a lot better and match more closely the uncompressed or the losslessly compressed image But it is again important to realize the distinction that if you lossy compress something Then if you overdo it you can throw away data that you did not intend to do Once that data is thrown away unless you have the original copy somewhere Then that data is gone. You cannot recover that data that you have thrown away in this lossy compression So this is in fact sort of a An important takeaway from this is that if you end up modifying say your jpeg files Let's say you take files Photos with your camera and you take them in the jpeg file formats You like to do a lot of modifications to your to your images One of the things I recommend is in fact saving an original copy of that jpeg file Because once you make some modifications and save that file and make some more modifications And save that file save it as a new jpeg file and and keep doing that cycle over and over You can in eventually degrade the image and this is not the same thing as opening a file And then saving it without actually performing any modifications the compression is It's it's not sort of it's not random in the sense that it's just going to pick random things to throw away You will in fact sort of get The same file over and over but it is sort of a warning that Through lossy compression. You can degrade the image over time in certain circumstances. Yeah If I would save the original file in whatever format the camera gave it to you as so if you If you download the image and it's jpeg that's as original An image as you possibly can get out of your digital camera If you're going to modify your image a lot, which you certainly can and should do then you can do that But just work on a copy of that file don't work on the original one just in case you do want to revert back It's going to become very difficult Especially in a jpeg once you make even small modifications to a jpeg file to go back to the original image There's i in fact And we talk a lot about this in e7 one of the things I recommend most in fact is in digital camera to take the Raw files just because that is that is completely like untouched It's not even there's a lot of processing that occurs to an image in a digital camera even beyond this jpeg compression to create A jpeg image within within a digital camera And saving the raw file allows you even more leeway in the data that is preserved But that's sort of an advanced An advanced thing where you don't really if all you're doing is just taking photos downloading them and sending them to your Friends and family and that's not something you necessarily have to worry about it Only when you start doing lots of modifications to your image or even some modifications to your images Do one of these other file types become quite handy Did I see a question somewhere? Nope, okay So we've talked about a couple of different file types. So let's then Put a perhaps a little bit more Data behind each of these and so realize that there is a type column here You can see that it's either raster or vector. I've I've been using the term bitmap to mean raster It's really kind of the same the same idea the same concept in this case So a raster type is basically just one of these images that is takes a Bitmap or a map of individual pixels and perhaps compresses it in some ways These are perhaps some of the more common ones that you would see for each type So in bitmap section, we would have jpeg jiff and ping And recall that there is in fact a bitmap file type and that is a very literal file that ends in dot bmp Usually windows machines work with it, but that is pretty much completely uncompressed And it is exactly what it sounds like that is pretty much the raw most raw bitmap that you can possibly attain from an image and But that's not shown here just because that's uncompressed and you don't typically see those All that often these days so jpeg for example, that's the one that we've been talking about This is the most common file type that you will see for images But it is in fact lossy compressed So this what this means is that the file size will be very small indeed And it will be very fast and very easy for us to transfer an image from From my digital camera to my computer to my friends and family But we are in fact throwing away some data for this to happen and compare this to say ping Or jiff which are not in fact lossy compressed. They're losslessly compressed But notice that jiff even though it's losslessly compressed you might say oh, I might want to use jiff for my images Realize that that may not might not be such a good choice Just because of the quantity the bit depth of the individual pixels in uh in the jiff There's only eight bits what that means again Is that you can basically only have 256 discrete colors within that image and and to be a little bit more clear While the image itself can only display 256 different colors. You can actually choose a palette So you can actually pick which 256 it will actually display But still we are in fact limited to 256 discrete colors within one image in a jiff file And ping sounds like it's sort of the best of all worlds right because it is not only 24 bits so it will actually display the colors in an image properly But it's also losslessly compressed the problem with this is that lossless Pretty much always is going to result in a file that is bigger than a lossy compressed Just because with lossy you can throw away bits that you don't need and you can compress it very very small You can compress a very large file down to a very small file And so we then uh get ping files for the same image are going to be a bit larger than their jpeg counterparts Now there's a final column on the far right called alpha. This basically just means whether or not there's transparency within an image The all images are are rectangular that you see on a computer. So all of the images that you've seen I mean, especially out of your digital camera. They're obviously rectangular so on and so forth But you can actually make them appear as though they're say circular by making some subsets of those pixels transparent so that then whatever's underneath will actually shine through That's all that alpha means alpha just means if there is some transparency available in this file type So jiff does actually support transparency and this was very famous or very well Used file type on the internet for a long time mostly because of this transparency issue You could just you could do some interesting things where you could create say A border that had a rounded corner for example And then you could make around that corner transparent And so on a web page that jiff file would look as though it were creating the sort of nice rounded border On on that web page nowadays you will tend to see ping Used instead just because now it does support a lot of colors. It's lossless and it also supports alpha But the tricky thing about ping is that some older browsers do not actually display the transparency Properly and so the final file type here eps. This is one that you're not going to see very much But it is an example of a vector file type It is something that if you were to download a vector file It might come in the eps file format now, uh, let's see so with each of these realize that All of this this is all the file the different types of files that we can actually store an image in and so we can actually store an image Of arbitrary size say 300 pixels by 200 pixels as it was the case of those flags or even an eight megapixel image in any of these file types here And really depends on the size So the reason that you would pick a different file type you you generally want to pick the one That's going to be most appropriate For the output of your image if you are using an image from a digital camera Then most likely you're going to want to use jpeg just because that's very very popular pretty much everybody That has access to any sort of computing device even phones have the capability to even I don't even mean fancy phones I mean like simple phones generally have the capability to display jpeg files And compare this to some of the other file types that are perhaps not well suited for something like Digital photos, but there is one more thing that we should talk about with regards to images And that is the size of the image itself So we've talked about how before when you can you can have an eight megapixel image and you can display it on your screen And if you display it at 100 percent and your screen resolution is not quite as large as the resolution of the image Then what's going to happen is you're going to have this annoying size difference You'll see only a corner of that image And you're going to have to zoom out a bit to be able to make that image display properly on your screen Well realize that it's actually very useful very important thing Whenever you download an image on your from your say digital camera from the internet And you want to send that along to somebody else Generally the nice thing to do the good thing to do is to resize that image so that it's appropriate For somebody to look at and generally this applies mostly to digital cameras because digital cameras tend to have Way too many megapixels that you know people don't even use most of the time Especially if what you're doing is sending an image from your computer to your friend or to your family Then what the best thing to do is to resize that image so that when they look at it, it doesn't inundate them It's not huge on their screen, but perhaps is size to fit a little bit better And so while you're not going to obviously be able to target a resolution Because you're not going to know what everybody's resolution is going to be Generally having something in the range of say a thousand pixels to maybe two thousand pixels wide Is generally a good target for your image if you're going to send it out And so the reason for that is that most displays these days are in the range of on the lower end of 1024 by 768 As is in the case here and they go all the way up to 2000 or so on the very large You know 24 30 inch display monitors and so that's sort of a good resolution to pick When you're trying to decide how to resize your image and send it out to somebody else Just because then they're most likely going to be able to view that image without that annoying Without having that annoying Way too large for the screen and the other upside is that resizing the image Actually goes a long way to modifying its size on a disc as well So it will also upload more quickly and it will download more quickly From your computer to your friends or family's computer Now there's a variety of other multimedia types as well And in fact audio is one that has been sort of controversial Especially a few years back and it continues to this day where Variety of file types have actually Made it very easy for us to store audio in a very very small way So uncompressed audio actually takes up quite a lot of space. I think it's Uh, let's see. What is it? So uh CD has about 800 or 700 megabytes or so of of space on it and they're generally about 60 to 70 minutes So we could say then generally that in uncompressed audio CD quality is going to be about 10 megabytes per minute And that's quite a lot. So if you start saving Some songs onto your computer and they're just made maybe three four minute songs something like that Then we're talking about 30 or 40 megabytes just for one song And this is uncompressed again And so there's the same idea of having different types of files that are compressed in different ways So we can then store these same files in a much smaller way on our computers And the most popular one obviously is mp3. This is the one that everybody knows about But this is sort of analogous to the jpeg in the in audio world just because it's very very popular pretty much Every computing device can interpret and play an mp3 file and it's lossy compressed It throws away data that the encoder doesn't think we're going to be able to hear And so it just does this based on some heuristics based on some research that the company that created mp3 decided Okay, well, we're not going to actually need to play sounds in this frequency or within this range or sounds that that look Or sounds that appear in this sort of In this sort of range are just not going to be necessary Because most people are not going to be able to perceive them or hear them It throws away those bits and it makes those those sound files much smaller So there's a variety of different file types wave wav aiff Those are all uncompressed file types and those are going to be absolutely monstrous massive lossy compressed mp3 is one of them Aac is another one. Apple tends to use quite a lot of aac in their own In itunes and in their own Itunes store and those are in fact, I think generally for the same size I believe people say that aac tends to have slightly better sound quality than mp3 for the same size file losslessly compressed These are not quite so popular just because they still result in quite large ones But flak flac is is a pretty popular one Aac also does have in as a losslessly compression a losslessly compressed format That you can do but generally when we see aac it is in fact lossy compressed And there's some other less popular ones like aug which is in fact lossy compressed and some other ones that That do in fact make up variety of audio formats But realize that there's even more than just these sorts of formats where we can take In actual songs some actual sound and compress it down to some Some smaller format and then the computer will actually be responsible for decompressing it And interpreting the bits and playing back the original song file itself They're also there used to be especially in the earlier days of the internet A much simpler type of sound file and I realized that this is not quite as visually exciting to look at But bear with me for a second. We do have two sample sound files here One of them is a file type mp3. I think or it's at least some some sort of regular Compressed format that we are familiar with and what this does is it's just the it's just a segment of Mozart's 40th I believe if I remember correctly And it sounds like this it's basically what we would expect from an mp3 file It just it sounds like it's an orchestra Playing it's some recording of an orchestra that is played at some point and we've compressed it so that we can then hear it at a later point Okay, so this is common. This is nothing new hopefully to all of you if all of if any of you are new to mp3 I recommend opening up your web browser and looking around on the internet It is very popular these days now one of the older formats though It was a midi file and a midi file was actually very very small It was very it was not even compressed very much But just because it didn't have to be in fact if we take a look at the difference in size You might be actually kind of surprised so realize that we have okay, so it wasn't in fact mp3 compressed it was compressed using aac The symphony number 40 the one on the bottom and then the one on top is in fact the midi file format We can see the difference in size 15 megabytes as opposed to 90 Kilobytes and so I don't I always get annoyed when People put things they always put comparisons of numbers in different In different powers like this. So just to make this real. This is 90 kilobytes and we have 15 it was 15 right 15 1,200 Kilobytes so just to give you an idea of the difference in size We're talking about quite a lot, but you must be saying well, okay We probably have to sacrifice quite a bit of data then in order to achieve this compression And in fact realize that this is not in fact a compression But what it is is something else entirely it is in fact just Instructions to the computer telling it to synthesize the notes It's just a collection of of instructions just a collection of notes within this file that says okay play this Chord and then play this chord and this instrument You should play an instrument that sounds like a violin that plays this note at this time that sort of thing So it basically is sort of like the equivalent of sheet music for a computer And even though a computer is then able to create sound like this It obviously sounds synthesized And so this was something that was very popular back in days So i'll just let you listen for a couple of seconds so you can sort of get the idea Unless any of you are tearing your ears out by now just because it sounds so awful Well, this was in fact something that was like I said popular early on in the days In the days of the internet just because this was how somebody could create some music and send it to other people Because 90 kilobytes is a lot easier to send than 15 megabytes of data Especially over old old dial-up connections. So it downloaded much more quickly and it was much more popular But obviously this Didn't take off in modern times for obvious reasons. It's just not there's no support for vocals There's no way that Even within different computers that it's going guaranteed to sound the same In fact, if you put this on your own computer, your computer might synthesize the sounds a little bit differently It might sound better might sound worse really depends on your own media synthesizers so realize that there are in fact other types of of media as well so Video now is a sort of a compilation of all of these things that we've been talking about It's obviously a collection of pictures that are then displayed one after the other to give you the sense of of motion And then it also generally tends to include things like like audio as well. Well Video is actually a pretty it's a pretty complex topic. It actually builds upon a variety of things that we've talked about now And so a video file generally is just a container. So it looks sort of like this Well, it doesn't look like this, but this is just sort of an analogous representation. What a video file might be It's just a collection of files contained within it You're not going to be able to see what these files inside of it are necessarily what they're named because they're given a very strict They're they're named very strictly and they're not really it's not really this idea of having Files that you could put into a video file and take out But this analogy does in fact stand where you could have a package or more Or just this sort of wrapper that exists around a variety of different files And who's in conglomerate makes up this video file So one of these things might be a sound file It could be an mp3 or it could be an aac file That's used to represent the soundtrack of the movie or actually the all of the voices and the sounds that make up This video file or this movie might actually be within this sound file that's contained within this container Then you might have another Another thing altogether, which is the video itself And so you might be familiar with some of the names of some of these wrappers like a dot mov file for example Or a dot avi all of these are just names of wrappers that that's or packages containers that contain a variety Of other files within them so avi is one of them for example mov is a quick time file type mkv is sort of kind of popular these days Let's see rm, which luckily we're not using very much anymore For real real media real player that sort of stuff And all of these contain within them Probably something like a kind of a you can think of it like a video file and kind of like an audio file as well And so each of these wrappers then might have some different type of file Contained within it and you might have heard of some of these codex these compressors decompressors that actually make up Some of the video files. So one of them is like divix for example another one that's very popular these days is h264 And these are just technologies that compress video itself And generally some or all of these packages can actually contain These different types of video encodings and so again, this is really just this idea of a container where this They define in some way in a movie file for example Where each of these these video portions and audio portions are going to Are going to exist within the image or within the the file itself. So this is the containers And this is the video codec And again a codec is just a compressor Excuse me compressor decompressor. It's just a way that A computer is able to compress the video into a known format Just like we have this concept of of run length encoding in jit versus this sort of blocking format that that they use in jpeg And there's also audio support as well. And generally you see common audio types like mp3 Or aac or some other different audio types that might be In existence aug for example aug the aura and all of these might be in combination with some portion With some video that's compressed and say h264 In combination with perhaps some soundtrack that's compressed in mp3 We can actually fit both of those in some file That's labeled mov So what does this mean then why why is there all of this complexity? Well realize that if we just have A video that looks sort of like this And we have we actually look at the contents of that video We can see a couple of interesting things realize that looking at the format of this video We see a lot of really interesting things about this file format itself. So this is in fact cars 2 trailer and if we look at the source Of this portion right up here the very sort of the first line that's in bold right there We can see that it is in fact a movie container dot mov So this is a quick time file that contains within it Some audio and some video just because if we start playing this we will in fact see that it is Playing some Audio and video after the this thing plays for a second. So here all right Right audio and video so you can see both at the same time So there's obviously something that combines this audio and this video and that is what we are looking at Right here. So within this movie file this dot mov file We have a couple of things and those couple of things we can see in format. One is h264 That is the video codec for the video that's actually playing on your screen We can actually see some other information about it 1920 by 816 Common millions so based on the discussion that we've had so far today. What does this stuff mean? What does this information mean besides h264? What is 1920 by 1816? The resolution so this is in fact the number of pixels wide by the number of pixels tall that make up this particular image Now what about millions? Yeah colors, so that's how many colors we have so in fact most likely then this is a 24 bit color Because it has millions of colors and we have 1920 pixels across by 816 pixels down Now ac this is in fact like we talked about in audio format two channels We didn't really talk about channels, but that just means it's stereo left channel and a right channel You can see the uh, there's this mysterious hurts thing again, but again, this just references how many uh cycles per second Or how many of how many uh samples per second we actually have for this particular sound Now there's a couple of other interesting things as well fps. That's the number of frames That's the number of images that appear in this video file per second And generally for anything that is film related is is almost always going to be 24 frames per second anything that's film is going to be 24 Generally things on television are about 30 frames per second unless you happen to live In a country that uses the pal type, which is then I think 25 frames per second And so there's a couple of different Common frames per seconds that we might actually use now we can see the data size The the size of this entire thing is going to be about 152 megabytes with the data rate we can actually see the current size So this actually highlights what I had mentioned before what happens if we actually have Uh an image that's too big for our screen 1920 is too many pixels for the resolution of this projector Remember this projector was a thousand pixels across 1,024 pixels across which means that if we viewed this this video At full size we would be missing half of it just because half of it would be on screen and half of it would be off-screen So what we have to do then to actually display this video properly is to try to resize it to fit The screen so we can do that just like this Did I see a question before yes Oh on the blackboard under aac aug og g that's just a file type called aug theora. It's a less It's not as well known But still among geek circles sort of popular because it's sort of an open Source format I believe for audio as opposed to mp3 Which I think at least for a long time was closed source. They did not actually or it was at least It had restrictive licensing. I think was the problem and it's sort of a similar thing now with h264 and Google has recently come out with another file type themselves, which is Should I'm blinking on the on the acronym it's something like Vpm or something like that's where it actually it's supposed to be this License free version competitor to h264, which they're using now in their own chrome browser Any other questions So there's a couple of other wrappers as well that we might actually be familiar with one of them You might actually It's okay if you're not too familiar with it, but you have probably used it at least another one is swf Or the flash file type and this is just a container that can contain a variety of things one of them Some flash containers can actually contain h264 So if you go to youtube for example, you're watching some youtube videos Chances are you might actually be looking at a video that was encoded with h264 And probably either mp3 or aac for the audio but contained within an swf container Now there's other ones as well mp4 is sort of another common one is Almost interchangeable with mov these i'm just sort of listing these so that if you start to see these online They will hopefully make a little bit of sense realize that mp4 is sort of confusing it's sort of confusing because it's not Necessarily related to the mp3 audio. It's just meant as a container which can contain other formats Within it now when we're actually compressing video This actually can be quite a complex thing itself and so If we there's a couple of different methods that we can use we can have interframe compression Or we could have intraframe Compression and so the one that we use depends on you know, basically what it is that we're looking at So if we are looking at say interframe compression, then what we are going to be doing is compressing Between frames so between multiple frames. So what we're what we have here For example, it's an image of a b that sort of buzzes across A static background this might actually be a good way for us to compress if what if we can actually remove the background We can find out that between multiples of these frames in intraframe In rather interframe Compression then what we can do is notice that the background is actually identical. We can actually remove it We can actually say use the same background from the previous frame But now change only these pixels so that now the b appears So it is moving and we can recreate the original scene as shown at the very top But what we will do what the end result will be is that we can save Far fewer pixels just because we don't have to know then all of the bytes that we're used to make up The background of this particular video file Now another type of compression might be something that looks like this where it's um It basically just records the changes this sort of a similar idea We're just records the changes of each of the uh of each of the frames from the previous one So that it says okay Well based on this frame these these pixels change in some known way like this And so then the decompressor then has to look at that and say okay I'm going to display this image and change these pixels and compressing video is actually far more important I would say than compressing audio or images just because and if we think about this for just a second if we use something That's about dvd resolution. So we have an image. That's about 720 pixels wide by about 480 pixels down And it's at a standard film resolution or standard film Film Frame rate per second of about 24 Multiply that by 3600 so what that means is that we have 24 of those 720 by 480 frames 24 times a second For every second in an hour Then uncompressed we're going to have about I did this math before 27 gigabytes of data more or less And that's quite a lot of data and that's not even hd if we start talking about hd Which uses something like 19 20 by 1080 times 24 frames per second times 3600 Then what we get is something much much bigger something like 170 gigabytes just for an hour And this is a lot of space just for an hour worth of a video on your computer And this doesn't even count the audio So just for the video itself we are talking about enormous quantities of space So being able to have good compression techniques is something that is pretty important And right now some of the most popular in fact h.264 is sort of the popular Way of creating hd video just because it is very good at some of these techniques that we're shown That we're showing here just because It's It does it relatively efficiently it can create a relatively small file and maintain very very high Resolution and in fact looking at this The same trailer once again for cars 2 we can see that it is in fact at hd resolution So realize that when people are talking about hd They really are just referring to a set of resolutions or a set of known resolutions That's video files should fit in order to be called various levels So there's a couple of different types of hd So sort of the lowest one is 480p Then there's another one of 720p and then there's another one with uh, or no not 1920 that's the Horizontal resolution 1080p So these are sort of the the common terminology that you might see for hd video and luckily This terminology Is used not only in in the context of televisions, but also in computers and in videos All over the place really and so If we are looking at this video this video here is actually fits within the 1080p resolution So the maximum resolution for each of these is as follows So for 1080p all of these actually represent the number of of horizontal lines So that's sort of the easiest way to remember this so 1080p we have a 1920 width By 1080 pixels down at a maximum 720p is I think 1280 By 720 and 480p tends to be something like Let's say I think it's about 854 or 720 Depending on the aspect ratio by 480 pixels and so 480p tends to be the resolution of dvd players Just to give you a concept of this whereas full hd, you know that terminology that people love to use Actually represents 1080p So this is full hd, but 720p is actually considered to be hd resolution as well So people go nuts over these resolutions. They say oh, yeah, this is such high resolution I can see all the detail on them. And yeah, you know, it's it's definitely an improvement over what we had in the past but honestly We've been running these sorts of resolutions on our computer screens for a long time now and only Recently has the technology sort of caught up so that we can now actually display video At that high resolution just because there's so much data if we're talking about 170 gigabytes or so Of data every hour just for video. That's a lot of data to process when displaying some video for us On the screen. And so there's a there's a good reason why hd has sort of been Elusive just because there's a lot of data for us to crunch So when I downloaded this trailer, I actually downloaded two versions. There was this version, which is the 1080p version and also the older version or rather the dvd quality version Which looks like this now realize that there's something kind of interesting here And that is that the dvd quality version Actually fits pretty well within this screen So this sort of just goes to show you that if you are looking online and you want to just Default to always using the highest resolution video available realize that that's you may not necessarily have to especially if you are on a laptop For example many laptops don't have high enough resolution to actually display 1080p on their screen without having this annoying problem of having to resize the image Which can be you know drain on your computer's resources It's just sort of unnecessary to display it if your computer or if your television does not have the the resolution the capability To display such a high resolution on its screen So yes, this does it is in fact a noticeable difference I think especially from the earlier forms of From dvd forms or from early sd as it's so-called Broadcast forms of television But you have to have also the technology to be able to display it I think properly so that you can actually take full advantage of all of the resolution of all of the pixels That's available to you in such a high data stream Any questions on any of that? Okay, so when so just to compare these two files We can actually look at this file as well and just notice that the size itself The size in pixels is actually much smaller. It's 848 as opposed to being 1920 by some number of pixels But we can also see that the data size itself is much much smaller as well So this is about 40 megabytes Compared that to the super hd version up here Which is 151 megabytes. So there's this trade-off Yes, you're getting a lot of additional detail in this higher resolution But you're also sacrificing quite a bit Of of size in this data and you can also take a look at this data rate This is the amount of data that you're that On average that your computer has to process in order to display this image And so what this is saying is that okay Every second it has to process 9.31 megabits So that it is able to show this this image on the screen That might not sound like very much but because h264 is so heavily compressed There's actually a lot of computation that it has to do on these 9.31 megabits and compare that again To the data rate here. It is just much smaller It's now about two megabits per second for this lower resolution version of this same thing And this is why if you're trying to watch video That's very high resolution on your computer and sort of Stuttering and there's it's dropping frames and it looks like it's very jittery Then this could be a Reason why your computer just may not have the capability to process so much data And display all of these pixels and go through all of this audio at the same time to display it to you at the end So with many now this sort of it's sort of nice now that a lot of tv's now Especially use the same sort of terminology because now we can start using this same terminology not only For our video files on our computer, but also on tv's as well when we're talking about hd resolution And all of these different types of of data Now finally, of course, um, there are all of these videos are in fact tend to be Encoded in some way, but they're also encrypted in some form Usually they have some form of digital rights management some dr. M in other words that are applied to it and this really is sort of Very it's an always Ever-present controversy it seems with media especially a few years ago The whole mp3 thing that that happened that went down and still to this day Is the recording industry still finding people that are sharing mp3 files and suing them for Lots, you know huge sums of money and now sort of the same thing is going down with the movie studios as well Where they're trying to find people that are sharing these movie files So just because it makes it very easy in fact to share these bites of data just to share all of these bits of information From one computer to the next Doesn't mean that you know, you'll be able to necessarily get away with it but just realize also I think it's sort of a An interesting concept that that's that will have to be resolved eventually I mean it makes all of our computers make this so easy for us to be able to Copy data from one computer to the next and so what is going to be the end result of this Of sort of all of this stuff that's going down between the riaa and the mpaa the um the Recording industry of america and the motion picture association of america respectively What it's actually going to happen with all this and it's something that we'll have to watch out for But just take care of if you are Online and you're trying to find downloads and videos make sure that you are not necessarily copyright You're not infringing anybody's copyright because that could be something that's sort of a silly bad thing for you to do Even though um, it seems like very many people are are doing it these days It's just sort of something to definitely watch out for and especially with on uh things like youtube There was a whole debacle a few years ago Where youtube got sued YouTube and its parent company. So now it's owned by google or sued by viacom for having placed some Some infringing copyright material up there even though youtube had nothing to do with it It was just their users and so what what constitutes sort of this? This acceptable use can users do this can Does youtube have to be responsible for policing this data? It's sort of an interesting thing and there's a lot of Controversy going on in other countries as well I think um France might have recently passed or they were considering passing a law that said, you know, you have basically three strikes It's this so-called three strikes law where if you're caught Downloading illegal content three times in a row. No trial. No nothing. You are without internet access They cannot legally give you internet access or something like that So it's going to be interesting I think to see how all of this plays out in the next few years and i'm not saying that This is necessarily something that the average person has to worry about But you know if you are doing this and it is something to watch out for if you are in fact Doing this sort of sharing of copyrighted content and for now that is all we will say But join us next week when we are going to start talking about security And start scaring you all again with some of the stuff that can happen on the internet and do your computers Until then thank you all for coming