 So, what do you want to talk about today? Well, yeah, that's a good question These slides took me a long time and I don't think it's worth the payoff, but let's find out. I was gonna say they don't look that good Hello, sir Hello Jake So I occasionally get comments on the video and people think I'm Surma and people think you are Jake So I think we we should sort that out and you see what I did there, right? I've called you by your name I don't usually shout my own name out But maybe maybe we could put a little thing along the bottom which makes it like super clear who's who because you know import for Like import for our individual brand identity. Anyway We've talked a lot about the the size of images, right and how to make them small But I want to dive a bit deeper and look at the different styles of loading an image when I see a page like this it feels like There's something missing, but I don't always know what I mean It's good on you that the space that there is a blank space because on many pages We don't even have blank page a blank space And you think oh this site is ready to read and you start reading and then things start moving around and it's really frustrating Oh, yes The absolute worst thing that can happen here is a layout shift But this site this site has at least left the space for something But is it a photo is an illustration is a graph is a piece of interactivity like it could be You know more than just an image So a smaller image at this point will reduce the time that the user is in this kind of what's going on phase But we can actually do something while the image is loading as well And that's what I wanted to focus on in this episode You might have seen some types do this I want to say are you going to talk about blur hash because that that had attention for a while Yes, so this is a blur hash or you know some other Form of making it clear. There's an image here. It could just be a couple of gradients or something like that And I would say when I see this it's clear to me now that there's supposed to be an image here It's probably not a graph, but it could have called content in it Like it could have text over the top that wouldn't be obvious from this view What I'm essentially saying is sometimes if there's a slow loading image like this a lot of time I don't care. I just going to read the content, but you only do that once you know the image is Just illustrative of the content like it's not a call. Yeah content in of itself You can solve this problem by throwing more bytes preview like this and once I see this I'm like, right. Okay. It's a kid kid with an ice cream Fair enough. It's just an you know, it's supplementary content so I could just go ahead and read the article and in the meantime the image can continue to load and That's a much faster user experience compared to like having nothing and then the image appearing You know fully loaded meanwhile users just staring at a blank area wondering what's going on But there are many ways to tackle this problem There are some solutions which are pretty new and there are some which are almost 30 years old And I'm going to try and cover all of them In a small video I'm going to start with the old one going to start with JPEG. That's the 30-year-old one, right? 29 I think but there have actually been innovations in this area even in the last few months I want to talk about those as well. So here's an image. That's pretty hard to compress for whatever reason I don't know why this is a really difficult image to compress I guess it's because there's lots of sharp lines Lots of different areas of the image different things going on. I mean technically it's probably quite easy to compressable just look Yes As I was trying to avoid that with this image and it turns out you have to throw a lot of bites at it in order for it to Not look terrible Especially in some newer image formats It's harder to compress this image in you a newer image formats, which is unusual but does happen But with JPEG they're both 300k. They look pretty good. They're both the same quality But they're quite different in how they're encoded Here's how they look when they're loaded over a 2g connection. I made it super slow so you can see what's going on now I'd say That the one on the right feels like it loaded faster Like you could tell what the image was a lot sooner In fact, it kind of feels like the one on the right has finished loading But it hasn't it's still filling in tiny details. These are you know, they take the same amount of time to load because they're the same size right There are people who don't like the effect on the right of it loading in multiple scans like that But I think they're wrong, but I mean don't don't you remember from I mean I certainly do from the dial-up days where most images not most but many images behave like the one on the left the Progressive not progressive, but they're just like with a line-by-line appearance of the final image and then your internet would cut out or Buffering what happened or something happened just when the the core part of there was about to have me like there's the The character I was waiting for something and then the image just stops loading. Nothing is more frustrating Exactly and whereas the one on the right which yeah is a progressive JPEG You get like this low quality version first and then it comes in and fills in the tiny details The one on the left is not progressive some software calls it baseline although that's sort of a slightly different thing But you know, they sometimes use that to mean and not progress as well a baseline JPEG can't be progressive But it also means it can't do some other things Anyway, Squoosh the image compressor that we work on that creates progressive JPEGs by defaults because we we like it We use Moz JPEG Mozilla's JPEG encoder Although there is an option in there to create the non-progressive kind if you really want for whatever reason Now I said there'd been an innovation here So I'm going to take a look at that Chrome and Firefox have received an update to their JPEG decoder recently So here's what it looks like compared to Safari, which which hasn't so three two one go And it might be difficult to notice the difference if you're watching this on like a small YouTube window or for you Soma who's watching it through a teleprompter But it's really noticeable at full-size like for real users here. It is up close. This is the first pass in Safari You get this blocky nearest neighbor Scaling whereas in Chrome and Firefox you get this much nicer smooth appearance and that's pretty new Like Chrome and Firefox were blocky until recently So it's the the blocky part. I think they call that the the DC Layer, right? It's literally just the because JPEG divides them into eight by eight blocks. Are those just the eight by eight blocks With one solid color basically. Yes, that's right It's the it's the DC part of the encoding. So it's solid colors And yes, what you're getting in Chrome or Firefox is just an interpolated Version of that. That's yeah, most Progressive JPEGs will have the first phase just being the DC information. So they're both basically the same data Just that Chrome and Firefox are applying a post-processing effect on the decoder side to make it look more pleasant while more data is being loaded Yes This is literally the same image and loading in each browser at the same speed. It's what you saw there Interesting a screen capture of it. Yeah, so Chrome and Firefox use LibJPEG Turbo as their decoder where Safari uses a custom one. I think a one that Apple made So it doesn't have this benefit yet They might add it. Also Safari's decoder sometimes just doesn't decode progressive images in a progressive way Like it just waits until the whole image is there and I don't know why it happens with some images and not others I don't know. I guess I guess it's some bugs But We also have bugs in our decoder and I found some bugs just making the slides But one of the reasons it took me so long to get these slides together is I started finding bugs in our decoder While making it so then had to go and file those bugs. We sometimes fall back to this blocky effect too early like where we shouldn't That's something we need to fix But JPEG is actually really amazing as a format because you can script like when you're encoding an image You can script what the progressive passes should be like how much detail each pass should have What detail it should be missing and how many passes it should have in total. It's really flexible We might actually ship our own in Squoosh That makes yeah, I saw you had like a draft PR open and I saw you know You were throwing like moving around some some c-code from the Mods JPEG encoder And I never realized and I know I ported the Mods JPEG encoder originally And I never realized that we had detailed control over the the individual passes as they are called of the JPEG decoding That's really really cool. Yeah, so what Mods JPEG will do is it it does the DC pass like those eight by eight blocks and then it as soon as possible Gives you like a low quality version, but with a bit more data And that's actually not great with the the new decoders that Firefox and Chroma using because it throws them into a sort of more blocky Output much earlier than we otherwise would so what I'm trying to do with the one in Squoosh is to avoid that mid step But still deliver You know a sharp image as soon as possible So I'm not going to ship that yet because I want to wait until we fix our decoder box Because then I can get to see exactly what the difference is But if it's still better, then then we'll ship that but yeah, I'll put a link to all of this stuff in the description All right, that's enough about JPEG I want to play a little game and it's all right guess the formats And I'm going to show you an image loading and it's your job to guess what image format it is okay It might be JPEG it might be something else because it could be a different form of JPEG anyway Here we go three two one go What you're thinking you still think you still there I I'm trying to think I think that might be PNG. Oh Well No, you're right. I'll give you that in one. That's this is the lesser spotted interlaced PNG so this is a feature of PNG that's Not used so much. It uses a form of interlacing called Adam 7 which means it seven scans it starts off with a one eighth resolution image a bit like the JPEG we saw before and then it doubles the horizontal resolution Then doubles the vertical then horizontal then the vertical and you end up with seven passes in total GIF has a similar thing But it's only four passes and it only improves the vertical resolution. It starts off with the full horizontal resolution But what I will say is Don't do this don't use interlaced PNG's it makes I mean you've been looking at PNG compression and JPEG XL compression recently how it uses the the pixel before to do prediction about the next pixel This form of interlacing makes that very hard So it ends up being about 20% bigger. It's not great for compression Generally PNG should be quite small So they don't really benefit from this interlacing thing. I would say If you're targeting an old browser Where you need the alpha transparency, but otherwise it's full of the data. So it's a huge PNG Maybe do the interlacing thing But otherwise no Squoosh doesn't do this right now, but I have a draft PR ready to go Do you know the reason I didn't like push this PR your way is I thought it would give you a clue to this question I was hoping you would get it wrong, but you got it right anyway But we'll ship interlaced PNG is in Squoosh off by default of course Nice, I like it. Okay Another question here we go. I'm ready three two one go Ah There you go Okay, so there was no Progressive step in that sense. It was full resolution from the very beginning just top-to-bottom. It has alpha transparency So there was some form of progression like it you you saw some intermediate steps, but as you say it was It was just a top-to-bottom scan. It was just a scan line basically. So yes This has proper alpha transparency, not just a mask. So and the amount of colors I'd say give us out JPEG is out and it Could be PNG it could be a VIF I guess it could also be JPEG XL, but I actually don't know I think JPEG XL progressive is the whole point that they're marketing is it's the same progressive rendering as JPEG So it would probably actually start with a low resolution I am going to say It's gonna be JPEG or AVIF Not a PNG or AVIF PNG or AVIF. I'm not sure how I would distinguish those to adjust by the Progressive render. I'm just gonna put my money on AVIF because you haven't had that yet It's a reasonable bet, but in this case, this is a lossy I forgot about webp The one you didn't mention. I think if you thought of webp you would have guessed it was but it is specifically Lossy webp so yeah top-to-bottom scan no fancy progressive rendering like JPEG has Part of this is because JPEG is an image format Whereas webp is an image format created from and the keyframes of a video format a VP8 and video formats Do not need multi-scan progressive rendering. They just need to display one frame at a time So it's not something webp can do in fact from what I'm told It was actually a lot of effort for them to even make the top-to-bottom rendering work It was yeah, they had to do a lot of shifting around of data to make it work But there was another element of the load there, which is interesting What I'm going to do here is I'm going to compare lossy webp with lossless webp Lossless webp is like a whole of a format. It's not related to the video codec at all. I'm going to start them now Now the lossy version finishes first because it's a smaller file But notice how it took a lot longer to get started compared to the lossless version Yeah Now have a hunch go on then. What's your hunch? I think You know lossy webp is lossy But one thing that I have learned Throughout our work on Scrooge is that it's very hard to lossy compress the alpha channel And so if you have lossy data You probably have to encode the alpha channel separately lossless because otherwise because if I know it's going to look really really weird That's that's correct. I would say correct answer. Maybe not entirely the right reasons okay, it's it's more of a product of it being a derived from a video format again What these formats tend to do is they you know they encode channels one by one And this was the thing that was difficult for webp to you know untangle So they were like delivering things together, but they didn't do that with the alpha data So the alpha data sits at the front of the file and you're correct It is lossless it uses lossless webp for the alpha data It sits right at the front So that weights that we had at the start was it loading all of the alpha data before it could start adding the color data Which is you know invisible exactly And Yes, I mean they could have done it the other way around but alpha data tends to be smaller than the the rest of the data Whereas lossless webp because it's a pixel by pixel compression. It does the alpha data along with each pixel They could have found a way to interleave the alpha data, but it especially would be in completely different format not easy and Lossless alpha does work something avi f does quite well But yes, that's not how webp does it. Oh, I didn't know I didn't know that avi f can compress loss loss fully loss fully Compress alpha and not have and deal with the artifacts that might happen It works really well So when people struggle if you've got like a lot of gradients and stuff in your alpha channel Whereas that's where avi f will do really really well. All right next format. Here we go three two one go any early guesses There we go So that is an image format that has no Progressive capabilities, that's correct Um Actually now that I think about it I don't even know why I was considering avi f because I think the whole point is the avi f currently doesn't do progressive because again It's a video format thing. It's probably avi f. You are correct. Yeah, this is it the avi f decoders in chrome and far fox They don't have any intermediate rendering at all. It's not even clear whether they can I think maybe the same problem again like the the two color channels are brought in first maybe it might not even be possible for them to ever do any kind of You know even scan line or progressive rendering here and I sped that up compared to the other demos we've seen I Think avi f is magic. It's my favorite image format right now, especially because it's in chrome and will soon be in firefox I've written a big blog post about how great it is But there are some kinds of images that it struggles with and this is one of them I had to throw so many bites at this image To prevent like ugly flat areas appearing it ended up bigger than the jpeg in this case It's one of those outliers Not usually how it goes. It's usually a fraction of the size of the jpeg sometimes like a tenth of the size and it still looks okay Not not the case here That's the whole point of scoosh where you can try these things out because I think one of the things that we really want to Get people to understand like there is not a single image format to rule them all like There's different types of images where different image formats excel at and so it's often the case of actually taking the time At the very at the important images on your website the big ones actually take the time and look at What scoosh has to offer in the different formats and use the ones that work best for that specific image? Absolutely? Absolutely? So I this way that you get nothing until you get everything with an avi f I've been proposing a way to mitigate this. I know you've seen this already It looks like this So this is a four point three k avi f put with a blur filter over the top So it's a tiny little image that can sit at the front especially compared to like 300k for the full image now Without the blur it looks awful Kind of very sort of alien almost art in some way But yeah, once you apply the blur I think it looks like a really good preview and the avi standard already allows multiple images to be in one Container and those can be tagged as like thumbnails So what we're going to look at is you know, if there's a thumbnail to start the image We could show that early in the browser and then like maybe apply this blur filter to it All just ideas right now, but I'm hoping we get something like that So you'll you'll get a preview like this straight away and then You'll get the final image wants it wants it downloads And I mean, yeah, if if the the final image is 300k Adding another 4k isn't really that big of a deal. It's an increase by 1% So your users can get a preview of the image much much earlier, especially for avi f Like if you think you're you know, you're on 3g Downing 4k will be significantly faster than downloading 300k So the time difference will the user sees something will be massive Absolutely, and I'll put links to the spec discussion and some more demos of this effect in the description All right, one more example. Here we go. Three two one go Little gap at the start Oh, and there we go And you know, I don't know if you can see this you might not see it on the smaller screen But we've got kind of these I can see that this yeah blocks appearing of higher resolution it exactly uh, so this is like a a two phase progression sort of like you get the low detailed dc t Similar to jpeg the eight by eight squares and then it's going and filling in The the the full resolution block by block This was a jpeg xl That would have been my guess. Well, it's the one left because that's the one there was. Yeah This doesn't work in any browser yet jpeg xl is behind a flag in chrome and in firefox, but there's no progressive rendering yet A little bit like jpeg. There's multiple ways That progressive rendering can be done and that can be done at the in-code time But also the decoder has has some say in the matter as well Um, this was the settings I was recommended it can do more passes and different kinds of passes Yeah, currently in chrome and firefox it loads more like an avi avi f you get nothing and then you get the whole thing But jpeg xl has been designed with this progressive rendering in mind So hopefully that's what we'll get when it lands in in browsers proper So where does this put us? we've got these Three formats that have some kind of progressive nature. We've got jpeg We've got jpeg xl and then we've got the Haki idea of the avi f with an extra image at the start and not png. Oh, I'm not including that rubbish The image is so big. It's not worth it. Uh, I'm gonna load them all at the same time. Here we go three two one go jpeg still does really really well here um jpeg xl gets to full detail a lot faster because jpeg xl was the the I could get To a decent quality with a much lower file size with jpeg jpeg xl in this case Um in the way that I couldn't with avi f it did a much better job of it Um, but then yeah, so jpeg xl gets to full quality. Um quicker than jpeg but jpeg got to the lower quality quicker than jpeg xl And that's really tiny avi f with the blur filter that was that was there first But it takes forever for it to get to the the full version just because it's such a huge image in this case I'm really excited about the different innovation in this area in different directions that this is taking My gut feeling is still that I will be probably using avi f for most Images on the web Because I'm I'm quite happy with the kind of loss that I get with avi f like it smooths images a lot, but in most cases um That a bit like the the image I started with like the kid eating ice cream You don't need full detail. You just need to give You need to not look ugly But you just need to give the impression of what the image is as soon as possible and avi f tends to do really really well with that but The philosophy of jpeg xl tends to be more for like big high quality images. So things like unsplash flicker if you've got a A news website that you've got a dedicated images page then maybe jpeg xl is going to be the better thing to put there Um, and you get this lovely progressive rendering to go along with it, which is I'm really excited about that I mean I'm actually also quite excited to see If and how web p2 will Fair in the battle of progressive rendering because one thing For me at least that web That sets web p a part is jpeg xl was designed with the web in mind avi f wasn't designed for the web at all it was designed for video Web p is not designed with the web in mind, but specifically for the web They made a whole bunch of shortcuts with like we are trying to find a format For the web jpeg xl wants to work on the web but also work in other use cases And I think that different scoping will allow them to Have a whole bunch take different trade-offs than what jpeg xl As you were saying like they're actually jpeg xl does really well at high quality images in my experience Um, not necessarily at the let's Make it look good with less bytes at screen size necessarily like they do there well sometimes as well But not all the time avf in my experience does better in that scenario But I do wonder if web p2 will Make interesting trade-offs and become a contestant here. Yeah, and it's really too early to tell with web pv2 It's it's all sort of hype right now, although we do have it in squoosh a preview But I do like the way they're heading they're heading like what can we do? That's similar to avif but with all of these web benefits Uh on top and if they deliver on that Then my favorite will switch from avif to web pv2 if they deliver on that because I know they're looking at all of the progressive stuff as well so it's Yeah, we'll do another episode on it When when there's something to show there, but it's it's inclusion people will have a play but it's it's super experimental right now In the meantime, there are other techniques. I showed you this page earlier, but with a blurry preview You mentioned blur hash. I'll put a link to that That's a way that you can create these previews in just a few bytes in this case. It's 50 bytes but You also need to ship a decoder for that which is kind of Fine if you're a native app because you can just put that along with your apk or whatever But on the web if it's your main image Then shipping a decoder along with it becomes or you have to factor it in certainly Um, it's only 1k. It's super small 1k with broadly compression but If you're talking 1k and then a few bytes or the images themselves You know, here's what a 1k webp looks like with a blur filter over the top Like you get much more detail for around the same size if you factor in the size of the decoder And also, you know, it's it's javascript reliant It can potentially break or not even run for some people and it will probably, you know It will occupy the main thread while this kind of image decoding can happen off the main thread There are lots of trades of involved in shipping your own image format versus using something that the browser already supports Yeah, and in terms of trade-off, it's worth mentioning that these blur effects are not free either They can actually be pretty expensive. So that's always something to to measure as well versus like, you know What kind of trade-off are you making there with that with that blur filter? So it's not something I would use a lot on a page But yes, like really one of the best ways you can deal with this thing right now Is just use a format with a nice progressive render. If it's a big image Jpeg is right there right now in squoosh. You can use this progressive rendering stuff Um, and yeah, it's an old image format But sometimes it will give you that best result especially for for you know images that are like 100k 200k If you can get down to 20 30k with avi f then it's that's a better option. But yeah I find it really interesting as you say jpeg being close to 30 years old and actually still it is literally competitive to The next gen formats in some scenarios because it's designed for the web um I'll also put a link to a discussion where the largest contentful paint metric is is looking at How they should consider a progressive image rendering because they don't right now So I'll put a link to that because they're they're going to consider some kind of That cutoff point an amount of detail is good enough um So that will be very important when we start getting more progressive formats, especially things like jpeg excel, but That's all I wanted to cover I think that was a good episode jake. Oh, thanks. I enjoyed it. I learned something Hey, do you know what as long as you press the little thumbs up icon that's that's all I care about It's gonna be one one thumbs up 800 down But now I've said that it's going to encourage people to click it Do you know never mind edit this bit out just don't don't show this bit lucas cut this bit out cut this bit out Definitely cut this bit out. Bye Bye We've talked a lot about the the size of images, right and how to make them small Okay, as you said, I was wondering where you're going with ice cream, but Yes Well, probably don't have that slide on the screen yet. So I mean that's not to lucas Whether this reference works on that it's going well Uh, but I want to dive a bit deeper and look at the different styles of loading an image