 Hello everyone we are on time so I guess we are on time for lunch as well I'll try to wrap it up so that you know we are not delayed for that and I am Rahul I'm co-founder at imagekit.io it's a relatively new product I'm sure not a lot of you would have heard about it but then what we basically do is real-time image optimization and transformation and today I'm going to talk about something which as you know a lot of people have come up to us and ask that you know how do we optimize images based on the user's network right let's say if the user is on 3G or if he is on 2G how do we change the image that we are sending to them and consider typical case where you know an e-commerce website loads hundreds of images on the website they would want to do some kind of optimization for the users who are on a slower network so that is what we are going to discuss that how do you optimize images on the basis of what network the user is on and yes I am from imagekit and I would be using imagekit because you know this is a sponsored talk you would have read in your schedule but then what we discuss can you can implement that on your own as well you can use any in-house solution that you might have for images you can use any other third party that you have for images and not just images you can in fact apply what we discuss to other more critical parts of the application which is a JavaScript and the CSS you can optimize those as well on the basis of the network so let's get started this is a rather interesting start and I guess most of you you know who have worked on front end would be aware about this that all the new users that are coming on the internet are coming through mobile right and globally that's 55% of all users that are coming from mobile in India that's a slightly higher percentage that's 78% mainly because of cheaper handsets, cheaper data plans coming in with Geo and Airtel and why am I talking about mobile right when I'm talking about service workers because network speed is actually a problem on mobile devices and not you know you guys work in your offices and you have a very stable Wi-Fi or a LAN connection you do not have network problems there that network problem is on your mobile devices for example if you step outside your building you move into the lift lobby or let's say you move into basement parking your network speeds would drop right so our ideal target audience for this is people who are using mobile and that's a big chunk for all websites right and we often believe that you know with Geo and Airtel and everyone coming in our users you know they have a great 4G speed or a great 4G network but the fact is not right we did this study with one of the companies that we work with what we did was that Chrome exposes the downlink speed the estimated downlink speed that the user is observing at that time and we tried to measure that we did that for a lot of users over a million users and this is what we see that almost 45% of the users have a speed lower than 2 Mbps 2 Mbps and in fact almost 25% of users have a speed less than 1 Mbps so you can so we assume that our users have a really fast network but then the reality is that they do not right and contrary to this in the last 6 years our page sizes have gone up by 3 times and the reason is that you know we started pushing a lot more JavaScript code with the client side rendering happening we start pushing in lot to the client videos have gone up in the last last year right so our page sizes have increased our assumption that the network has improved stays but then the reality is that the networks have not improved as much and the result is that our users end up with a frustrated experience they might not even come back to the website right I'll go back to the previous slide I just wanted to focus on the green bar that is there at the bottom that basically corresponds to images and I would like you to notice that images still correspond to about 50% of the page width right videos are coming up but then images are still the major component in the pages right so this this makes it very clear right that up until now we have been able to modify our website and our content on the basis of let's say the user's DPR or let's say the user's device width we have made responsive websites but then not a lot of us have you know worked on optimizing the website on the basis of the network speed and that must change right we need to stop building for ourselves we need to stop building for people who are there on a wifi connection and can load our website really fast we need to start thinking of users who are not on a very good stable connection and we need to start building for them right and our target in the stock is we are going to be focusing on images reason being that like like we observe that they still form more than 50% of the page width plus the other reason being images are the lowest hanging fruit to be honest right they might not have as much impact on the overall page load time as you know optimizing JS or CSS would have but then that they are the easiest ones to optimize and they are the easiest ones to start off with and but then you can apply this concept to any other content on your page as well right so now we have two requirements one is that we need to determine what network the user is on and detect when that network changes and the second requirement is that when the network changes we need to change the image size that we are sending to them right so we look at the first part right how do we detect network changes so Chrome and Chrome mobile that is what I recall from what I last read have implemented this network information API what this does is it observes all the downloads that are happening in the browser and it tries to estimate what kind of network the user is on right and it exposes a read-only value which can be read from navigator dot connection dot effective type it basically classifies the user into four buckets 4g 3g 2g or slow 2g so when I say 4g it's basically speeds upwards of 3 mbps right that's the average it includes Wi-Fi as well so even though the classification is 4g and 3g 4g would include really good speeds including Wi-Fi as well right so we do know what what network the user is on what kind of speeds is he getting right and 4g obviously is faster than 3g 3g is faster than 2g and 2g is faster than slow 2g so the logic dictates that right on 4g which is the fastest network of all we need to serve the largest image size and on slow 2g which is the slowest of them all we will serve the smallest image size right that's that's what we want to do now how do we change image size right we're not talking about changing the image dimensions we're not saying that we will resize the image and send a smaller image to a slow 2g user what we want to do is to mince you know compress it further right compress the exact same image further and for those of you who would have worked on images on the front end you would know that there is a factor called image quality that is associated with an image so very simply put image quality is a scale of 100 100 being the best quality one being the lowest quality 100 being the you know the sharpest and the best image that you can get so I've taken this image as an example right let's say I have the image quality set at 90 which is a standard for a lot of websites they set their image quality at 90 the size is 47 kb right let's assume that we will use this for the 4g network next let's say for 3g network we use quality 70 which is giving us a 40% reduction in size now assume that you have a website which loads one MB of images if you are able to save 400 kb out of it you have actually saved a lot for the user who is on 3g similarly for 2g we will use a smaller quality level 55 for slow 2g we will use 40 and for slow 2g we are actually reducing the size of images by almost 72% right that's a huge reduction for the user right lot lesser data to download so how do you change quality this is where image kit comes in right we provide a lot of transformations that you can do in real time on an image so for example let's say I have this image URL image31.jpg and let's say I want to get a quality 90 image all that I am doing here is I am appending a query parameter PR equal to Q-90 it gets me a quality 90 image if I change that to 60 it will give me a quality 60 image if I change that to 10 it will give me a Q10 image and I am not sure if you can see it on the screen or not but then the Q10 images is actually really low in quality if you especially notice the clouds above the house you would find that those are really pixelated and that is the reason why the size is small right so apart from quality image kit does a lot of other transformations but for now I will stick to quality right that is what we are trying to optimize so we have figured out two parts we know what network the user is on we have figured out a way how to change the quality in real time you can use anything apart from image kit if you can do that right I am just using image kit as an example now how do we handle all of this in our code right the idea is that we do not want to be changing our code right we cannot have our server sending different URLs for different networks please we do not want those kinds of changes right so this is where the service worker comes in service worker is I guess a lot of you would have worked on service workers or would have worked with service workers right service worker sits as a intermediate proxy between the user's browser the cache storage in the browser and the network right and what the service worker allows us to do is it allows us to intercept every request and then decide where do we get the response from do we get it from the storage do we get it from the network do we modify the URL and that modify the URL part is what we are looking at right so what we will do is let's say I have in my html let's say I am using image 1.jpeg if I detect 4G in my service worker I will change the URL to use quality 90 if I find 3G I will change it to use quality 70 on 2G we will use 55 on slow 2G 40 right this is what we will use the service worker for we will intercept the request and we will change the URL from where we are getting the response so let's look at some code my image tag remains exactly the same I am not changing the image tag my quality parameter does not come in the image tag our image tag remains the same next part is obviously we install a service worker I won't go into the depths of it it's a pretty standard thing you just need to call service worker.register the file name and it gets registered now this is the main function right service worker allows us to hook on to an event called the fetch event this fetch event is basically fired for every request that is originating from your browser what we do in the first two lines is that we it's a very basic check we check if it's a JPEG image or a PNG or a J for a WebP right if it is we continue further we check the connection type for that connection type we determine what quality do we need to use and in line number 11 in line number 13 in fact we create the new URL the new URL basically would be we have appended the quality parameter let's say to that URL and in line 14 to 17 instead of responding with the same URL we respond with the modified URL so for example if the browser sent a request for image 1.jpeg and we modified it to let's say to use Q70 instead of so so we respond with Q70 we do not respond with the original URL right so let's look at an example and on how does this work so so this is a basic you know something which we have created similar to an e-commerce website I've already loaded it but then this has been loaded without without the service worker coming into picture what I've done is I have throttled the network to fast 3G this is something that you can do in Chrome and I will just reload the page once and I hope that the demo works it does so what I have done is I have also overlaid the quality number at which the images are being fetched on right so use if you see all images have been fetched on quality 70 right all of them have this number 70 overlaid in the top left corner and if you see the network tab this is the original request right this is the request that the browser fired I was resizing it to 300 by 450 the service worker intercepts it it issues a new request which has Q70 as one of the parameters because the service worker detected that the user is actually on a 3G network right so this is this is what the service worker was able to accomplish right so this this is very good to begin with right so we were able to detect the network and we were able to modify the image we were able to send a lighter image to the user but then there is a problem with this right and the problem is that the browser does not know that you have modified the URL in the service worker right for the browser it's a black box if the browser requested for image 1.jpeg and you responded with let's say quality 40 in the image the browser does not know and the browser would cast that low quality response forever or for as long as the cache control allows it to do it right we do not want this to happen since we are modifying the cache on the basis of network we do not want the browser to behave as a black box and cash everything that we are giving to it right so one one thing that we need to do is that we will skip the browser cache completely right instead what we will do is we will use the cache storage API which is there in in all modern browsers I guess in most of them and we will use it to create caches for different network ties for example there would be a different cache for 4G there would be a different cache for 3G 2G and slow 2G and what we will do is we will implement a waterfall kind of a lookup wherein we always look for the best resource we know that 4G has the best images it has images on quality 90 so we would always prefer looking for an image in the 4G image cache then we will go to 3G then we will go to 2G and slow 2G right this is how we will structure our cache lookups this is something that you cannot attain in a browser that is why we need to skip the browser cache completely so let's look at the code if you remember from the previous previous code example we were responding with the the the transformed URL so instead of that instead of responding with the transformed URL we have a new function which is a step down cache lookup we will we will do a cache lookup and if that cache lookup fails then we will go to the network right this is the cache lookup code I will give I will simplify this for you and I have shared the code on github as well I will share the link on where you can read the code I will just simplify it for you line number 10 9 and 10 is basically just looking for an object in the cache let's say in the 4G cache if you find it in the 4G cache well and good we return it from there if not we go to the next cache which is 3G right if you find it there we will return it from the 3G cache if let's say we do not find it from 3G cache and we look up let's say in the 2G cache and let's say the user is on 2G right now if if the object is not there in the 2G cache as well then we go to the network which is line number 16 and 17 we go to the network if we have a successful response we add it to the corresponding network cache which is being done in line number 18 right so now we have four different caches we are adding objects to different caches on the basis of network right and we are also picking up the best object that we have available at that time if we have 4G available we will use 4G object so let's see how does this actually work out in a demo right so let me just check if I have cleared all caches I have I'll start with a slow 3G right so this should get me a quality 50 image ideally because it's slow 2G everything is slow so you can see the first image which loaded its quality 50 I'll update this to fast 3G and I'll reload the page again now because the network has improved the browser automatically fetch the better variant of the image it fetch the quality 70 image let's say I remove the throttling completely and I reload the page again don't tell me it didn't work so basically it takes time for the network connection type to update so like I mentioned earlier the network information API it looks at the download times that the user has been observing and because I was observing 3G like download times it kept me in that 3G bracket for slightly longer now that I have moved to 4G now we get the 90 quality image now let's do the reverse let's say the network downgrades again let's say the user falls back to slow 3G what would happen right we still have the best image in cache and it should ideally just pick that up it picks up quality 90 image and if I look at the cache storage so we have four different cache storages here is this visible to everyone at the back the cache storages so we have a 4G network cache it has all the images cached in 4G we have a 2G cache we have a slow 2G slow TG we never access slow 2G so the cache is empty we have a cache in IKIT in the 3G cache as well but then 4G is the preferred one so our images would always be picked up from this right so basically we have implemented a way where as soon as the user's network speed upgrades the images would upgrade as well but then let's say if the user speeds goes down we would look for the best image that we can serve to them and you know not downgrade his experience now this presents another problem which is cache management earlier the browser was managing the cache for us now we are adding objects to the cache so we also need to ensure that we are clearing the objects from the cache for this I have used Google Workbox you can figure out any other way to do it what Google Workbox allows is that you can set up rules on every cache either on the basis of the number of objects that are there in the cache or on the on the time that for example let's say objects in the 4G cache would remain there for let's say 30 days and I can have a maximum of 1000 objects in that cache right I can set up this rule and I can create what is called an expiration manager right so this is what we do to manage our cache we instantiate an expiration manager the last part of the code this is how an expiration manager is defined using Workbox we specify the max age in seconds we specify the maximum cache entries that can be there and in addition to adding the object to the cache we also update the timestamp when the object was added in the cache that's been done on line number 6 right so that you know the expiration manager can see how long this object has been in cache and from time to time we keep expiring the entries you can you can do a very simple set interval kind of a thing every 60 seconds you just clear out all the objects that should not be there right so that's a very simple way of managing what stays in the cache so to summarize one thing is very clear it has been a long time that you know we have been optimizing for different devices different screen sizes it's time that we start optimizing for different network speeds as well because to be honest network speeds are not that great in India and we must stop considering that you know we are accessing our websites from our offices or from our Wi-Fi's for us everything works blazing fast right but then that's not the case for our users so we need to start thinking about them we need to start optimizing on the network speed and we talked about images but then you can get the gist right you can extend the same concept to JS and CSS you can basically tell your server that okay the user is on a 3G network send me a lighter version of the website right so this this can be extended to beyond images as well right we needed two parts of the problem one is to detect the network changes and to maintain different caches for every network for this we have the network information API the service workers and the cache storage the other part on how do we modify images in real time that is what we do at image get from quality we have real time resizing cropping transformation etc face detection overlays it comes integrated with the CDN so the performance stays right up there and you can integrate it with your infrastructure as well right we can talk about image kit later but then as I said you can do this even without image kit you can do this with your own in-house image management solution if you can modify quality in real time or you can use it with any other third party image product the demo page which I showed it's there on this link service were swopt one and the code is there on github in our image kits github it's a very basic code I haven't had time to add the readme file I'll do it as soon after the talk and you can contact me either via email or follow me on twitter if you have any questions that's it thank you so my question is in general also in context to image kit as a service to say I'm on a slow network and I have images which have been loaded in poor quality right but as a developer I want my user so my images to be upgraded so what I'm talking about is progressive enhancement right like so is image kit giving me something or in general there's a solution for this so what we discussed was a very general solution right if I see that the network has upgraded that let's say the earlier the user was on a 2G network and you loaded a low quality image but then the network upgraded to 4G the service worker automatically fires a new URL a new request right so the images automatically get updated right so that depends on your application logic right what you want to configure I configured it in a way where it upgrades instantly let's say you do not want that to happen let's say you want let's say you will wait for let's say 5 minutes before the user is on a better network to make a new request you can implement that in a service worker right it's a very generic thing it's not related to image kit like I said it's something that you can change in your service worker itself that do you want to instantly upgrade to a better image or do you want to wait for some time or you know on some other logic do you want to upgrade to a better image right it's something that you can do in the service worker itself hi Rahul where? here right here can you see me here? okay yeah hi Rahul my name is Shashank so your talk was quite informative so I am working on couple of products in which I have to support users on IE 6, 7 and 8 so you have to explore 6, 7 and 8 good luck with that sir that's all I can say I mean in the best case scenario see if you if you go back to the code the image tag basically doesn't change and if let's say the browser does not support service workers even then the same experience gets delivered to your user so in any case the experience is not getting hampered the only thing is that if the browser does support service worker then we do something additional otherwise the experience on IE etc would remain the same the image tag would load as it is right is there any alternatives because I have seen couple of GitHub projects where there are couple of alternatives but the options are a lot I cannot hear actually how about now is it clear? not yet can you hear me now? yeah okay so I was just trying to fix this and on GitHub there are lot of basically repositories which have this solution but as in the solution there are lot of products out there so I'm finding it difficult to choose the right one lot of products for network based optimization or image optimization for replacement for service workers for IE 8 and 7 you mean something like a polyfill for service workers right? yes that's right I haven't exploded to be honest I mean I haven't come across these polyfill so I cannot comment on it as well but then if there are maybe I'll look into it and maybe I'll update my presentation and talk as well thank you hey hi so Chrome provides this API to deter the network like 2G or 3G so what about Firefox, Safari and mobile applications so as of now the network information of the API is available only in Chrome and Chrome mobile which is basically almost 70% of the users accessing normal websites right but then I will update my presentation I recently came across an implementation which is done in service workers so basically what the network information API does that it is calculating the effective downlink on the basis of the download speeds that the user is observing the service worker does the same thing right so your support for network information kinds of get extended to other browsers which support service workers right so that's a slightly wider range of browsers than just Chrome and Chrome mobile but then beyond that like I said it's a progressive enhancement it's not something that you know if the browser does not support let's say network information or service workers the experience still remains the same but if it does support we do something better with it so this is a basic elementary question so can the same approach be extended to switch the format of the image so if I'm using Chrome and my browser supports WebP so can I intercept a PNG image and instead server WebP right so in ImageKit in fact we do that automatically right so that is why I didn't cover it because in ImageKit when we see that a browser or a device supports WebP we automatically switch it without even changing the URL right but then yes it can be extended to other browsers if you can detect that the browser supports WebP you know through content negotiation as such you can modify the URL to fetch a WebP image as well so there are no restrictions around the like if the result MIME type differs from the requested image format so there are no restrictions around that so for example you would have seen cases where in the URL ends with .php but then the response is image slash jpeg the content type is a jpeg image so that works perfectly fine because the browser is actually looking at the content type header and in fact the actual content that is being sent back and it tries to determine how to display that content on its own so it doesn't matter what the original URL has in the extension for WebP in fact you don't need to actually go to the service worker it's a very it's a much simpler way in which you can you know implement WebP which I can talk about you know after this talk it's not related to this but then you do not actually need to go to the extent of using service workers for it Thank you Hi, I have a question can I use this ImageKit IO if my image is not uploaded on the your domain Yes, you can integrate it with your own storage or your own image server anything right so we'll get it from there we'll do everything in real time and send it back Okay, thank you Hi, I have a question as you said that network type is returning 4G or 3G string right I'm not able to hear it again that network type is returning 3G or 4G or 4G right like that so what happens if 4G is returned but actual speed is actual downlink speed is low so there are two observations which I had when I was actually working on this yes there are cases where the actual downlink speed is low and the browser still reports it at 4G so in fact if you look at the code that we have uploaded in GitHub we take that into account that if the speed is less than 1 megabits per second we do not use the effective type that is being returned by the browser we automatically set it to 3G that is one way of doing it that you look at the downlink speed instead of the effective type that is being returned by the browser but then there is I read it in another place that these downlink speeds and their mapping to the effective type it depends on it varies from browser to browser and it can change in future releases as the network speeds improve the browsers would update their method of tagging as well so for example what is being tagged as 3G today it may not be be tagged as 3G let's say 2 years down the line so either you can rely on how the browser is tagging it or in certain exceptional cases like we have done if the downlink speed is really low then we do not actually rely on what the browser is using to tag that so it's something that you would need to take a call on it right if you want to go with the downlink speed or if you want to go with what the browser is telling you the effective type is can you hear me? can you hear me? can you just quickly run us through how imagekit would integrate with my infrastructure because I think in most use cases I would serve it from my infrastructure images offline ok cool thanks that's it? yeah hi so does this imagekit works only for images or it works for 3D models as well for network optimization? it works for more than images as well but then again not in the scope of the presentation so maybe we can catch up outside and you know we can talk about it ok so since it improves the network optimization by compressing the image size right what if we pass the data out of the image and show some business data when you compress the images or models the data are being lost right? ok so you are saying when I compressing the image the data is being lost yes but then see it's a trade-off that you need to decide on on the basis of your application I think showing a slightly lower quality image let's say which is 30 to 40 percent smaller but then at least showing that image to the user with let's say an option for him to be able to fetch the original image I guess that is still much better than not loading the image at all now imagine a e-commerce website where the image does not load at all versus you are at least able to show let's say a slightly lower quality variant but then where the user still gets an idea that ok this is a black t-shirt that you know that I'm looking at so if the user gets some idea I guess a fair bit of you know reduction in quality obviously you would lose out on detail that's a part of it but then the user still gets a fair idea of what is the object that he's looking at and I guess that turns out to be favorable for a lot of use cases but then again it depends on your application you may want not to go for it at all we pass the data of the 3d models as well as the 2d models sorry we pass the data based on the emails or the 3d files we pass it and show the business data to the end user for us it will be not relevant to that right I didn't get your question sorry to be honest hello yeah so for us we pass the data out of 2d or 3d models and show the business data to the end users ok so for this if you optimize this and reduce and compress the data and send it to the end user it won't be that much efficient for the end users maybe I mean that depends on what application you have like I said so this is very effective for a lot of use cases but then maybe not in your cases ok thank you yeah hi so how the lazy loading of images how is that difference and or should it be used in combination with image optimization so lazy loading of images is basically you're not loading something which is not visible on the screen but then when that image is visible on the screen you still load the highest quality variant right you do not take into account that the user is on let's say a 2g network you still attempt to fetch a larger version of the image that is what the difference is versus network based optimization that you know lazy loading is something different lazy loading is let's say you will load load 10 images and leave the rest of the 90 out but then network based optimization is that when you're loading those 10 images you would take into account that the user is on a 2g network and that you would load a lighter variant for him and not load the you know the best quality image so they're different concepts actually ok hi so my question is on the lines on her question I'm here oh right in front of you ok yeah so say for example I have the image as you know lazy loading I create a hook after like you know 50 or 80% of the scroll the next 10 images should load ok but if the speed is too low sorry if the speed is too low can I have the service worker like change the quality and how much time does it take for the browser to understand that the network has changed and they need to like you know right so ok so basically there is no lag in terms of what the browser tells you the network type is right so it's instantaneous when you were refreshing something happened and I was like ok it was slow because actually the page was still being fetched from the server right so the page load is still slow all the JavaScript that is being loaded that is also slow only the images have changed so that is why I said that you know images is the easiest bit but then the actual performance gain would come when you implement this thing for your pages and your JS and CSS that is where the performance bottlenecks usually are right so images though is the easiest bit in terms of if there is a delay when the service worker is modifying the request or not to be honest I haven't seen any significant delay it's probably just a few, a couple of milliseconds extra where the service worker is actually modifying the URL but then I guess it's a very small time and it doesn't really matter I mean the user on a 2G network would probably end up loading an image in 4 or 5 seconds if we can make him load it in 3 seconds by serving a lighter variant so that kind of compensates for the extra time that has been spent in the service worker one more question so say if you are implementing this service worker for image optimization you've seen how Medium does it they send a blurry image which is very low in size so that the user can get an idea it's loading so would you choose that over this? so are you saying something similar to what we see on let's say Medium.com loading their images see it's again a call that you can make on the basis of your business if you want to serve let's say a really blurred out image yes that would be the smallest image that you can serve to the user but then that's a call that you need to make that whether it suits your business or not for an e-commerce company it might not suit the business for a content company it may be fine if let's say you are loading the image of an actor let's say in an article it's fine if that image shows blurred up initially but then that might not be true for an e-commerce image wherein you are loading let's say the image of a t-shirt you would still want to show the actual object in a slightly more detail instead of loading a blurred image over there that's again depends on your business logic what do you want to achieve?