 Hello, mate. Hello. This is like remote two or three, isn't it? It's good. Yeah, it's weird. It's like we have almost all the technology that we need to make this work without feeling different, but it's still different. We'll let other people be the judge of that if it works or not. That's true. So I'm here to talk about image compression. It's something I've been wanting to talk about for a long time, right? I mean, we wrote an entire app around it. We did, and we are going to be seeing that as part of this talk. Nice. I like it. If you look at the studies they do across web pages, like images are the biggest thing on the page. 50% roundabout of each page is just the 50% of the bytes that go over the wire for any given page are on average images. Yeah, and sometimes it's fun. I'll go and take just like some images from a web page and like recompress them. And usually I can halve the size of them without losing any of the visual fidelity. So it's just like so much stuff on the web that can just be compressed better either by using a different format, a better encoder, or just like better settings within a particular encoder. Yeah, that's what I found as well. I've recently done a little experiment and found like even without like going into like the very deep settings of an encoder, often people just don't optimize their images at all. And just like doing the default setting in Scooch will already give you a really decent compression rate without losing visual fidelity. So if you're watching this at home, if you're the kind of person who just like saves your images out of Photoshop for the web, then this is the talk for you because I have things to discuss because I think you're missing out on huge performance gains. So I want to dive into image codecs. I wanna show how they work and I wanna explore some of the lesser known settings. So here are the formats, right? We've got Lossy, JPEG and WebP. We've got Lossless, PNG, GIF and also WebP. Hang on my friends, hang on. You put GIFs and a Lossless. They are Lossless. They are like- With their 256 colors? Yes, but you make that change before compression. Like the GIF compressor won't accept images more than 256 colors. That's a step you have to run before the compression. It's Lossless, trust me. Because we are doing the same for PNG as well. But I've got a question for you. If you have a digital drawing, like a drawing that has been created on a computer, which format would you use for this? If at all possible, SVG. Yeah, I agree with you. Because a vector can be any resolution, so it's ideal for drawings. But sometimes the drawing is really complex. So they're actually smaller and much faster as a Lossless image. But then again, if the drawing has a lot of gradients and shading, it might actually be smaller and acceptable in a Lossy format without it looking bad to the user. So a lot of the talks and articles I see about image compression, they try and bucket kinds of images into formats, and I think that can be misleading. So instead, what I want to do is I want to talk about what these formats actually spend their bytes on. So Lossy images, it's like sharp edges. They really struggle with that. That will increase the size of the image. And also small details in undetailed areas. That sounds confusing. So we'll look at a few examples of that in a moment. For Lossless, it's color change, especially unpredictable color change. So another thing, we'll look at some examples of that. And also total number of colors impacts the file size as well. With Vector, number of shapes and also shape complexity, there's a little difference with Vector and that's the complexity of the shapes. We'll actually introduce a CPU cost in a way it doesn't have the other formats. So that's something to bear in mind. That's the TLDR of this talk, really. That's it done. Well, thanks for doing this and I'll see you next time. Bye-bye. No, no, I want to get to know these codecs in detail. I'm going to dive in. Well, just to be clear, these are categories of codecs, as you said. Like each of these categories has multiple codecs to offer. And I think the basic advice should be that you just try the same image. Like if you don't know which category your image falls in, just try the image in all of these codecs. That's kind of the ideal process, right? Yes, but you can make some educated guesses based on knowledge of how these things work. So, Serma, have a look at this. I know this is difficult because you're watching this over WebRTC so some of these demos are slightly limited. Do you notice anything about this picture? Yeah, if it's not the stream compression, then I'd say the clouds have blocky artifacts going about. I think that is your stream compression, so this is... Okay, then let's try this again. When you say notice, do you want me to find something that is wrong with the image or just something that should help me make a guess on which codec is best for this? Well, I'll tell you what, if you haven't noticed it straight away, then that's good enough for me. So if at home, if you're watching this in full HD, that's 1920 by 1080, obviously you're getting video compression on top of that, so you're not seeing exactly what I'm seeing, but this picture, the brightness data is full HD, 1920 by 1080, but the color data in this image is 96 by 54. Like that's 0.3% of the data and you barely notice. Close up, you can start to see it, a bit of discoloration around some of the fencing. One of the fence posts has turned into a ghost. That's the most obvious bit. Yeah, like if the colors are, if you have that little resolution for the colors, the colors will start bleeding across boundaries, even though the light data still gives the contour. So that's, but yeah, from here it's not really, yeah. And this is because you've got bad eyes, Surma. Like you have these terrible human eyes that are really bad at seeing changes in color, but you're actually pretty good at seeing changes in brightness though. So I'm going to take this image and I'm going to flip it around so that the brightness detail is low resolution, but the color detail is high resolution. And here it is. And the first thing you'll notice is it looks bad, but it is the same amount of data just flipped around. Here, it looks fine. You can't go this extreme in all examples. So here's a image that has a bit more complex data around color. You can see this yellow stripe and the red stripe on the car. It's like there's some of the details quite blocky, but as we start increasing the amount of color data, again, it becomes difficult to notice. With this image, once I get the color data to around 5% of the brightness data, it becomes impossible to notice for me. I actually built a little web app to research this. I had a lot of- Of course you did. This is the first WebGL I ever wrote, Surma. I hope that- Oh, you did WebGL, that's fun. Yes, to do the color conversion. I'll put a link to this. So you can throw your own images at it. You can get some cool effects. You can see the individual color channels because this is what Lossy Codecs do. They call it chroma subsampling. It rather than RGB, they do brightness and then two chroma channels, two color channels. And that means they can just discard loads of that color data and save a lot of file sizes as a result. As you mentioned earlier, you get some color bleeding as a result, but it's barely noticeable, especially in like photographic imagery. One more thing that Lossy Codecs do in terms of Lossy stuff. It's another quiz for you, Surma. I've made an alteration to this image. Can you see what I have done? There is a very ominous circle between the trees. Yes, I've drawn a circle in the sky. There it is. But that's not all I've done. Down here, I've replaced the brightness data with noise. Just noise. But you don't really notice that at first. And here's the interesting thing. Here's a diff of the image versus the original image. Now, the circle in the sky is so subtle, it barely shows up. In fact, if you're watching this with video compression, it might not be visible at all. I can see it right now. But the noise in the bottom, that it's like a huge difference. But again, you have terrible human eyes. You're good at seeing small changes in what we call low-frequency areas. Like the sky, there's not a lot going on. You spot that small difference, whereas you struggle to see the changes in the high-frequency areas. This reminds me a bit of the blind spot that we have in our eyes where our brain fills in the gap for us because I see the circle. I mean, you showed me the diff and fade it back to the full image. I saw the circle and then the shape of the circle started blending into the environment and now it looks like a bush. Like, I know the circle is there but I can't actually quite see it. So I think it's just my brain auto-correcting and I'm quite frustrated with my brain for doing that. It does. And this is something that Lossy Image Codecs, they take advantage of this. So what they do is they'll take an image like this and they'll divide it into eight by eight blocks. And rather than describe these blocks pixel by pixel, they use this. They multiply these shapes together to form exactly the same image. Like not at that intensity. Each block can be at a different intensity. It can even be negative. But you can reconstruct any eight by eight image using these shapes. And this blows my mind. Like I don't believe this. I still don't believe this even though I built an app to show it. Look, I've made a little pixel art 203 for HTTP 203. I'm not very good at pixel art but that's what I tried to do here. But what you see below is all of those shapes that the intensity that it uses to reconstruct that image. So we can actually sort of apply them one by one. And you can see as it does the low frequency things, it starts to build up like this blurry picture. And then as it goes and starts to do the high frequency bits of data, the full image comes into view. It's incredible to me that this actually works, but it doesn't save much data really. But let's actually take a real bit of the image. This is the picture of the woods from before. I'm going to pick up some of the grass. Now what Lottie Codex will try and do is they'll try and discard some of this data. So you can see already it's not using all of those shapes. That's where these zeros are coming from. But as I drop the quality, and this is like JPEG quality in this case, we can see more and more of those zeros appearing. And the thing about sequences of zeros, they compress really, really well. So we're left with an image that isn't 100% accurate but this is high frequency data. You don't really notice loads of data is saved. I mean, even here in the side-by-side view, it is hard to tell which pixels are incorrect. Exactly, but let's take a different example. So this time a curve, which might be part of like a logo or something. And in this case, even a very small drop in quality is introducing noise that is very noticeable, especially if the color is solid on either side of this curve, that's low frequency data. This small amount of noise is really noticeable. It's like the circle in the sky. You'll see it very, very easily. And that's why lossy compression is not very good at sharp lines next to solid color. So now if you look at a picture like this, which is a heavily compressed JPEG and zoomed in, you can see what it's made of. You can see the color bleed from the sub-sampling. You can see the eight-by-eight blocks. You can see those waves that it's using to construct that data. And that's it. You now know how lossy compression works. There's only two elements of loss in both JPEG and WebP. And that's it. You've seen them both now. The rest of it is just- I have a question because I'm trying to remember WebP, I think, also does this frequency-based transfer on the discrete cosine transform, as it's called, the conversion to this frequency patterns and you add them up. No, that's what we just saw. Yeah. I think WebP does it also at bigger block sizes. If they are low noise. But I'm not quite sure. I'm remembering that correctly. Yeah, so WebP is just better. So there's a few different things. Like you say, it can use different sized squares. It can also, when it starts a square, it can use a nearby square as a starting point. Also, JPEG has to use the same mathematical conversion for every eight-by-eight block in a given channel. Whereas WebP can have four. So it can have a different strategy for high-detailed areas versus low-detailed areas. And it has a lossless compression thing as well. It uses arithmetic compression rather than Huffman. And it has better post-ecode filters as well. So it is just better. It also supports alpha transparency as well, which JPEG doesn't. So why have I wasted my time talking about JPEG? Well, Safari does not support WebP. It's the only one browser that doesn't. So that's the only benefit of JPEG, really. We can still use WebP for browsers that support it. This is the picture element. So we're serving WebP for browsers that support it and falling back to JPEG for Safari and older browsers. There are also client hints to do it on the server side. I'm not gonna go into it. I'll put some links for that, if that's what you wanna do. So let's actually compress some images. Now, the compression tools in things like Photoshop are not very good, which is one of the reasons that we built this. Not just us two, we're part of a team. But yeah, Scrooge.app. This has the latest WebP build. It also has more JPEG, a really good JPEG encoder from Mozilla. So I'm gonna use the F1 image from before, because it was a bit of a tricky one to compress. First thing I'm gonna do is zoom it out. So it's the same size as it will appear on the site. So if you've got an image that's gonna be 1,000 CSS pixels wide, but you want it to look good on a high density screen, you want a 2,000 pixel wide image, but then you wanna zoom it back out so it's the size it will actually appear on the screen. Now, I'm gonna start with WebP in terms of compression. And all I'm gonna do, if you mentioned this before, just bring the quality down until it looks bad. And especially with a high density image like this, you will be surprised how low you can go on the quality. Like it's still really barely noticeable. Now, it's really tempting to do this, but please don't, like don't put your nose to the screen, don't zoom in. Because if you do that, it'll look bad. It's not gonna look good, right? The more you zoom in, the worse it will look like, oh, look at that, that's disgusting. But users are going to do that. I mean, honestly, considering that we are at, that we just removed 98% of the data and it still looks that good, even when zoomed in is kind of cool. It's, yeah, absolutely incredible. And so if anyone comes at you and says like, oh, I took one of the images from your website and I zoomed it in by a thousand percent and it looks bad, like just say, I don't know, okay, zoomer and ignore them because that is not what real users are going to do and you should be optimizing things for real users. So keep it roughly the size it will be on the site. I actually wanted to look into some of these advanced settings in WebP because when we first built Squoosh, I didn't know what these did. I just took what the code was and made UI for it. There's a lot, but I now know what they do. I did some research. The really interesting ones, auto adjusting the filter. This is a good thing to do. This really slows down the encoding time, but it improves the visual a lot. Now, the filter is actually a decoding filter. It's just what it's going to do to remove that blockiness, but it increases the encoding time because it tries to figure out by looking at the image what the best kind of filtering is going to be. But yeah, it's just a one bit flag, or not one bit, like a couple of bits. Three bits, I guess, if there's eight levels of filter sharpness. Yes, exactly. I think there might actually be more. I think it might actually go from 0 to 100, but it's probably a bit. The other one that I always knew, like, oh, this makes a big change to the image, but I don't know what it's doing. This is spatial noise shaping. Now, like I said, WebP can have these four different strategies. Did we misspell spatial? Oh, maybe. Ha ha ha ha! Ha ha ha! Oh, that's a classic one. Do you know what the amazing thing is? I've actually spelled it wrong in my notes for this video. Well, that's a buck report. Anyway, yeah, so the other option is spatial noise shaping. And so WebP can have these different strategies for different parts of the image. SNS, what that does is it changes the extremity between those different strategies. So higher SNS takes bites away from the stuff that least needs it and gives it to the stuff that needs it most. It's like Robin Hood for image compression, basically. In some images, turning that up to 100 has a positive result. It doesn't hear. 250 is actually pretty good for this image. Can you go too high and sense it like going higher makes the image worse again? Yeah, so, and with this image, it does go worse. Ah, interesting. So if I make it extreme here, it's giving more data to the road because that's where the blockiness is, but it starts to introduce, like it makes the tires of the car look bad. I see. Okay. You start to notice the artifacts there. Because it's taking data away from the high-frequency area, giving it to the low-frequency. The image of the woods that I had on before, actually SNS of 100 works really, really well because it takes lots of data away from the leaves and stuff that you can't see the noise and gives it to the area of the sky. So that works really well. We're gonna need a JPEG as well. So just for fun, I'm going to make the JPEG the same size as the WebP just to show the difference in quality of the compression here. So I have to take the same again, call it you write down till it's about 40 kilobytes. And I'm gonna do what I said not to do because, okay, this looks really bad to my eyes right now at this level, but obviously with video compression, it might not be as obvious. So I am gonna zoom it in. JPEG creates this horrible blocky effect across the road. And like I said, this is low-frequency data. So this is gonna be like the circle in the sky. It's just super obvious. It looks horrible. The yellow strip, the transition from the yellow strip to the road makes it really obvious where the data is lost. Yeah, you get to see the blockiness there. And this is where the filtering that WebP does is, yeah, it's much better. So yeah, I just have to increase the size of the JPEG until that horrible blockiness goes away. And at that point, it's like double the size, over double the size of the WebP. There are things you can do to improve it if you wanna spend a time, more advanced settings. The most interesting advanced setting here, I think, is the chroma subsampling. Yeah. This is what we discussed before, right? This is just reducing that color data, especially if you're on a high-density screen, you can get away with less. In fact, most of the time, I would say you can just turn this up to four. We saw earlier that this particular picture is kind of sensitive to color reduction. So three, but it was a good one there. But that knocks 10K off, so it's worth having. I also often combine it with the separate chroma quality so that you can basically introduce the blockiness into the colors, but not into the brightness, which is often much less. That's interesting. I've never had success with that. I've always tried that, and I've never, I've always been unhappy with the results, but yes, like mileage may vary, right? The option is there. So if people can play around with that. Absolutely. But yeah, I think it's worth in this case to serve both the WebP and the JPEG, because it's like half the size for the browsers that support it. But that's it. That's everything I really had to say about lossy images. Well, you said you had major research about the options. What exactly is pointless spec compliance? Pointless spec compliance removes the progressive encoding and some of the other things that according to a strict interpretation of the JPEG spec, it's not supposed to do that, but everyone supports it and it's good for file size. So I thought I would totally put you on the spot, but you actually know the answer. Well done. Well done. Yeah, do you know what? I didn't research a JPEG, but that was one of the ones that I knew from looking into the encoder there. So lossless, completely different world. A lossless image describes pixels one at a time from top to bottom, left to right. But rather than describe each color from scratch, it will use some of the surrounding pixels to make a prediction, and then it just encodes the difference. There's a number of strategies that you can use. It can look at the pixel to the left. It can look at the pixel above. It can use an average of some of those pixels. But here's the gotcha. It can't change the strategy for every pixel because that would just add too much data. Instead, it has to stick with a particular strategy until it decides to change strategy. Now, PNG can only change strategy at the start of a line. WebP can define 2D blocks of different strategies. But this is why lossless compressors can be really slow because it's just brute force going through all of these different strategies to figure out which one is best. Like trying all the combinations. But it means this compression works really well. If the pixel next to it is exactly the same, or the pixel above is exactly the same, because there's no difference then, really easy to compress. And it's why lossless formats struggle with photo data because there's lots of organically changing color. Oh, one more trick that lossless things have. If there's 256 colors or less, it just sticks those in a table and it can reference them by number. So rather than every time, describing the red, green, blue and alpha, it just says, it's color number five, everyone. There it is, pop that in there. And that compresses really, really well. So, difference between the formats then. WebP, not many people know about the lossless WebP format. It's like a completely different codec. In fact, it is a completely different codec. The only relation between the two is when you do lossy WebP with an alpha channel, it's using the lossless format for the alpha channel, pointless fact for you all. Oh, that's good to know. Yeah, it's just better. It's got more strategies. It's got the 2D thing going on. It compresses better. But you do need the PNG for Safari. That's all we're talking about. And just never use GIF. Just don't stop it, please. Never use it. It's just, it's bad at everything. It's just bad. So, stop using GIF for everything. But let's put this into practice. Back in Squoosh, this time I'm going to use something with a lot of flat colour. And I say thanks to Stephen Waller, who donated me some of his artwork to use here. He's an artist I used to work with. He makes these wonderful drawings. Apparently, these are his teammates that he worked with at the time. I didn't ask him about the bear, but fair enough. I would say the first thing to do here, same as before, zoom it out. Keep it the same size as it'll appear on the site. So the kind of scaling that you would actually use to use the high-density devices. And now I'm just going to go to use WebP, because it's the best, and I'm going to pick lossless. Yeah, completely different codec, whatever. Very few people know about it, but it's really good. We're straight down to 43K. There's this slight loss option, which I've never found very funny. It's great, isn't it? And what it will do is it will change the pixels to try and make them more predictable. I actually don't like the visual effect it introduces, but so I don't really use it, but your mileage may vary. It's fun to play with. But remember when I said earlier that these formats do really well with 256 colors or less. We're going to do that. We're going to reduce the palette down. Because it doesn't look like this image has a lot of colors, but because of the anti-aliasing around all of the curves, it's got lots of slight variations, blends of one color to another. I've also changed the codec to BrowserPNG, and that's just because it's really fast to compress, so it's going to be, I'm going to get a quick response when I'm changing the number of colors. I'm also going to turn off dithering. If you reduce the number of colors of something, you can get this banding effect that you can see here. Dithering tries to remove the banding by using pointillism or these dots to recreate the shading. It's super effective, but lossless deals well with flat color so it will increase the file size, so I would avoid it unless you really need it. Although I am impressed, I have compressed a couple of photographs to PNG with dithering and it will actually look, if you zoom out, if you have it small enough, it will actually look almost the same. Often it's imperceptible. Yeah, and there are some times that you need to do that. So now, the same as before, I'm just going to reduce the colors as much as possible until it looks bad. And this is one of the reasons we built Squoosh because it's so much nicer to do this with a visual response than getting a computer to try and figure it out because it's not going to do it as well. I'd say important advice, same as before, don't zoom in because it'll look bad. You can see the harsh edge here, but zoomed out, you barely notice it so it's fine and that's how users are going to see it. So there we go. So we've been able to reduce the colors down to 68 and so now if we're going to go and just switch back to WebP, turn the effort up, let it use all of its iterations to figure out the compression and boom, there we go, it's like 13K. So that's amazing. Isn't it, it's incredible how much that saves. For Safari, we need the PNG. PNG compressors are really varied. So we use OxyPNG, which is a pretty good one and that will take it down to what we get, 17K. That's a close second, pretty good. It's close and I would say, I would give a shout out to ImageOptim, which ships with OxyPNG, but also lots of other PNG compressors that we would like to add to Scrooge at some point, but we haven't yet and it brings the file size down even more. In fact, it's only 3K off the WebP. So in this case, I would probably not bother unless I had like a load of images on the page and that difference adds up. I would say just serve the PNG, it's fine. Like 3K, it's not worth the complexity. So yeah, rather than think of a compression format for a particular kind of image, like my takeaway is like, think about how the compression actually works and use that as your first guess. But as you said earlier, it's always worth trying it in something else. So I'm gonna look at a couple of edge cases to finish up on. Look at this, look at this, this is beautiful. This is SVG, believe it or not. I was about to say that it looks like one of these things that the CSS artists built with just like a billion divs and gradients and shadows and then a photorealistic thing turns out to just be CSS. But I guess SVG has. That's pretty much what is happening here, but with SVG. I think this was manually created, yeah, very similar. If I minify it, an SVG OMG, which is a SVG compression. Well, it's a GUI I made, but it's based on SVGO, which is a node app. I can get it down to 37K. But this is so complicated and uses so many filters. Like look at the jank there on the resizing and this is on a high-end MacBook. It is taking a ton of CPU to render that image. So over in Squoosh, Lossy WebP does a really good job of this like 53K. It's actually bigger than the SVG, but the saving in CPU makes this a better decision. And that's actually interesting because that's something that Squoosh doesn't really allow you to quantify, right? The saving in the CPU. Yes, exactly. Yeah, it'll be nice if we could find a way to do that somehow. But yeah, if it's struggling in, with this image, you notice how slow it loads and you notice in SVG OMG how slow it is to resize. So that's a good signal. The PNG equivalent, because we can't use JPEG for this because it's got alpha transparency. The PNG is 86K. And I've had to reduce the colors and that introduced a lot of banding. So I had to introduce dithering. And that's actually noticeable that you can see the dots. It's not great. But without the palette reduction, it's 300K. So in this case, you would kind of say, oh, I'm sorry, Safari users, you don't get quite as nice an image, you know? Yeah. But yeah, it's worth it for the file size. Although I guess you could also serve the SVG because like, I mean the picture elements, you specify the sources in order of priority. So you could put the weppy first and could put the SVG seconds. Yeah, you could decide like to say, well, you know, Safari users are likely to be on a high-end device. So the SVG is less bad, maybe, but it's still gonna cause jank to other things on the page, you know? It is like as a high-end MacBook, it's bad. But yes, you could make that judgment call. Like you could say the jank is better than the 300K or the banding or whatever. One more example I'll leave you with. This is another one of Steve's drawings. It has a lot of flat color. So my instinct for this would be to go for a lossless format, but there's also a lot of shadows, a lot of gradients as well. So again, Lossy WebP does a really good job of this. Like, it takes it down to 28K. If you go for a lossless format, even with the palette reduction, it's 118K and you actually get a lot of banding with it. So yeah, it's like, tools, not rules. That's my takeaway from this. Like think about the codec in terms of what they are good at. Think about how they work. And with that, go and make your images smaller. Please, please make your images smaller. I've actually enjoyed this. This has been like, should we do some more two or freeze like this? Should we do some remote two or freeze? Because I don't know when we're gonna get back to the office. I was planning to, absolutely. Well, we've got the equipment, so let's do it. Happy next time. Happy next time. Bye.