 My name is Rich and I'll be your host for today's presentation. Joining me, I have Damian, who's a partner engineer for Chrome Enterprise, and he'll be speaking about storage best practices for web apps. I also have Andres from our signage in Kiosk Partner Upshow, and he'll be talking about a case study that explains how PWA has made an impact on their business. For today's agenda, I'll start with a quick introduction of the Chrome Enterprise Recommended Program, and the technical community hour. After that, I'll hand it over to Damian, who will make a couple of announcements, and then jump into today's topic. Following Damian, Andres will present the case study, and lastly, I'll provide additional resources for PWAs and open up the virtual floor for questions. Today's Chrome Technical Community Hour is brought to you by the Chrome Enterprise Recommended Program, which is Google's partner program for third-party solutions that are optimized for Chrome OS, or integrated with Chrome browser. This webinar brings you the opportunity to engage with our team about new features and updates, enterprise development best practices, and our enterprise strategy. Now I'll kick it over to Damian, and he'll make some announcements and start with today's topic. Yeah, thank you, Rich. I'm going to present now. So can you see, I spring well? Yes, I can. Right. Well, yes, thank you to get started on today's presentation. I'm going to start with some announcements that are not necessarily related to today's topic. So the first thing is that starting from Chrome 117, so this is going to happen in September, we're going to be deprecating the unload event. This is an event that fires before the user leaves the page, usually, but it's been quite unstable, and it has other problems because it prevents some browser optimizations like the usage of the back-forward cache. So that's why it's going to be gradually deprecated starting in Chrome 117. You have all the information in this article to anticipate to this change to make sure that nothing breaks on your site and where are the alternatives to migrate. So please check that out. The other announcement is actually more like a reminder. A few years ago, we announced that we are going to be deprecating Chrome apps, so for those that have production Chrome applications, the recommendation is to migrate to any of the recommended technologies depending on your use case. It could be web applications or PWAs for most of you, and in some cases, you can also use Chrome extensions. We have updated the guidance around this in this link, so check that out. And actually, today, Andres is going to talk about their experience with their company app show to migrate from Chrome apps, so this is a good opportunity to see a real case. To help you in that migration, we have started last month a three-part series on PWAs or advanced web applications. Here is the link for the first session, which was delivered by Adriana Jara, so you can check that out. And today, we're going to have the second one, which is exclusively about storage. But next month, we're going to have one of the Chrome product managers talking about advanced APIs, also the new things that are coming. So please make sure to join that as well. I think Rich is going to be sending invites soon. So this is summary of the content that we will be covering today. We will start with a review of some of the storage mechanisms that have some limitations, but many of you might be using, they have been around for a while. Then we will cover two of our recommended approaches of our technologies. By the end, we're going to look at some of the APIs to access and manage the file system. And finally, I'm going to talk a little bit about the storage limits, which means how much you can store in different browsers and specifically in Chrome OS. As we said, we're going to have at the end App Show talking about their experience working with progressive web apps and especially with storage. So if you want to know more about their experience, you can check out this link, which is the case study that we have released. But Andres is going to cover a lot of the details of their implementation by the end. So please stay until the end so you can also watch that part. So without further ado, let's get started with today's topic, which is about storage in the web. So the main reasons why you would like to use storage in your web applications usually have to do with either performance or reliability, including being able to respond offline and also user experience. For example, you might want to even add some features to let the user store things in your app or things like that. There are different types of storage mechanisms that you can use. And sometimes developers have a hard time deciding which one is better for the use case. So hopefully today we're going to help you on that decision-making process. So to start, these are some of the storage mechanisms that are available and you might use them for different things, but they have some limitations, especially around performance and about the amount of storage that you can use. So they're not very recommended, right? For most of the operations that you need to do when storing large amounts of data. And here are some of the limitations that these different mechanisms have. So to get started, session storage is constrained to a single tab and can be useful to store small amounts of information, but it's only up to five megabytes. It's also synchronous. And as we will see, this is a disadvantage, especially in working with storage. Local storage should also be avoided. It's also quite limited. So it's up to five megabytes. It's synchronous and can only be accessed in the main thread. Cookies, as you know, have a lot of useful case studies, sorry, use cases, like for example, session management. You might be using cookies for several things, but for storage might not be the most convenient mechanism, especially because every time you send a request, you have to carry all the cookies in that HP request and that adds a lot of overhead. And also cookies are synchronous and they are available in the main thread only. And finally, there are a few of these technologies that have been deprecated already like the application cache or are on the path of deprecation like Web SQL. So these ones should be avoided as well. So as we have seen, these storage mechanisms have some limitations. And when you're building a modern web application or what we usually call as a progressive web app, you might require mechanisms that work asynchronously and are also available in the window and worker context. So let's review these concepts first. Some of you might already know about them, but let's take a quick look so we can then go over each of the examples. So first let's talk about this synchronous and asynchronous concept. So synchronous JavaScript code requires that each operation finishes executing completely before running the next line of code. This can be very slow, especially for operations that require input output like storage accessing the disk or even like fetching something from the network. So in this diagram, for example, if you start to make a storage operation, you need to finish it completely before you can start the next one, which can be for example, the fetch. And the fetch can have the same problem if it's synchronous. So an optimization to this technique is to use asynchronous functions and use asynchronous code in general. And that allows you to decouple the start of the operation from the completion, right? So you can start an operation, use for example, a callback or a promise and wait for the operation to be completed. So you can for example, process data. And this is actually an improvement in performance. And as you can see, if you compare with the previous one, you are freeing up the main thread model, which is a good thing. But in both of these cases, you're still occupying the main thread. It would be more, it would be less, but you're still. So we need something else in some scenarios to make sure that your application is going to be performant and responsive. And as a reminder, the browser and JavaScript is single thread, which means that the browser is going to use a single thread, which is the main thread, to execute the code that you are writing, to also handle the user inputs, to do things like garbage collection and to do painting operations in the screen, like rendering, for example. So you want to keep the work in the main thread as lightweight as possible. And one way of doing that is to use a different thread. And the way to do that in the web is by using workers. Two types of workers that are the most commonly used are web workers and service workers. So in the first case, web workers are usually implemented to allow for some very specific operations to be made and they live in the context of a page or a tab. When you close it up, the web worker is going to be killed. Service worker are different because once you install them, they're going to be listening to page events all the time and they can do things like intercepting network requests. So we want the storage mechanisms to be available in both contexts. In your page, so your page code can access them and you can do things like, for example, use some of the data to update the screen. But also you want them to be accessible in the secondary thread. So you can offload entire operations and also for cases like offline, right? So if you have a service worker, that's going to allow you to intercept requests and to make a decision on that request. For example, say, I want to check the cache and if this, I can respond with anything that's in the cache, I will do it even in offline scenarios, otherwise I'm going to the network, et cetera. So the two storage mechanisms that meet these requirements are the Cache API and Index TV. So we are going to focus mostly on these two APIs next. Here is a quick comparison between them. So the Cache API was designed to enable service workers to cache network requests. So it's very good to store things like documents, like JavaScript files, like CSS files, et cetera. And Index TV is useful for any other type of data, especially if you have something that you need, like structure data, or you need to search for some data in like a very small database. So that's what Index TV is good for. In practice, developers sometimes use these things interchangeably, so they can use the Cache API to store some structure data or Index TV to cache static resources, but the APIs were designed mostly for these use cases. So let's go deeper into the first example, which is the Cache API. So the Cache API is like a key value per map that allows you to store requests and response objects. So it basically represents the HTTP requests and responses. For that reason, they can contain any type of data that can be transferred via HTTP. And as we said before, it can be accessed from the window, from an iPhone, from a web worker, or a service worker as well. So here is a sample code showing how to store request response pair into the Cache API. This API is asynchronous and you can use promises on it, but for simplicity here, we're using await to make this code more readable, right? So in this case, what we are doing is to open a cache object and then we are making a fetch request on the network and storing the response in the cache. As we said before, this API can also be used without necessarily going to the network. So if you want, you can, for example, create your own JSON object and you need to wrap that into a response object and then you can store it in the cache API as well. But the most common use case and the reason why this API was mostly designed was for the service worker communication. To go through service workers to fetch some resources and then service workers can make decisions and say, for this type of URL, I'm going to respond with whatever is in the cache or I'm always going to the network to fetch the most recent or fresh resource. So you have a lot of different ways to implement this. There are different caching strategies and this is one example, right? So here is the strategy that's called stale while revalidate. In this one, what you are doing according to this diagram, you go from the page to the service worker and then the service worker is going to intercept that request and it's going to check on the cache if that request is available. If it's available, it can respond immediately and simultaneously update the cache. So it can get as complicated as you want and there are many of these different caching strategies that you can use. Here is the code for this one. So this is the code if you implement this manually. Here what we are doing is to use a fetch listener in the service worker to intercept requests. The next thing what we are doing is to open a cache object. Then we check if that request is in the cache and then we do two things simultaneously. So if the request is on the cache, we respond immediately so it's very fast. Simultaneously, we send a fetch request to the network and update the cache. And that way we can have a mix of performance and freshness at the same time. But implementing each of these caching strategies like the one in this example requires a lot of manual and error-prone code. So a way to actually get rid of that is to use a library and the Warbox library provides a lot of out-of-the-box caching strategies. So you don't really need to write the code for each of these strategies every time you want to use them. Afterwards, Andres is going to show how they have used this library in their project to be able to, for example, to cache multiple gigabytes of video. The library is quite extensible. And just to give you an idea of the previous code that was like very long and had a lot of different things to do and decisions to make can be implemented very easily with Warbox in this way. So here what you are doing is to import in two modules and then using a caching strategy to declaratively say, for this URL, I'm going to respond using the state where we validate. And that's it. And then you have multiple caching strategies for that. So I think that's pretty much what we are going to cover about the cache API. Let's now move to the second one, which the ones that we mentioned that are recommended technologies, which is IndexEV. IndexEV is actually more complex than the cache API. The cache API was mostly like a key value pair map. But in the IndexEV, you have actually a large scale, no SQL storage system. So the database allows you to store almost anything that you can there. And it supports things like transactions and also indexes to make everything more performant. So it's an actual mini database that you have on the client side to use. With IndexEV, you're going to manage different components. For example, you're going to create a database. You're going to create object stores which are like the same as tables, for example. And then you have things like indexes. You have transactions. You have course source. So many of the things that you might use on another type of databases are available in IndexEV as well. So here's the code sample of a very simple IndexEV operation. The first thing we do is to create a new database. And once that happens, you can use a callback, which is called onUpgradeNeeded. That's going to be called once the database is open for the first time. And then you can create the schema, for example. In that case, we are creating an object store with a primary key. And then we're also creating an index to make it easier to access data. And we are indicating two fields for the index. Once that's completed, you're going to use the onSuccess callback every time you want to run an operation on the database. For example, a transaction. In this case, we start by getting references to the object store and the index. And then we can, for example, create an object for a person and do an insert operation here. After that, we can retrieve that object that we just put in the database by using the primary key. And we can use also that callback to print the name on the console. So that's more or less how a typical operation works. By the end, once the transaction is finished, you can close the database connection and that's pretty much it. The problem with this API is that working on it can be a little bit complicated because you are using callbacks and things can get a little bit messy. So there is a library for that, which is called IDB, which is basically a wrapper. That's going to add promises to indexed. So if you want to use indexed and you like to use promises because it's simpler to use, I would really recommend to check this library. And the last group of APIs that we're going to talk about have to do with file management. So so far we talk about storage that you can use in the browser, but we are not really impacting the file system at all. In this case, if you have an application like let's say Visual Studio, for example, and you want to open a file, to show that file to the user, let them make changes and then save the file in the file system, you need to use something different. So that's how these file system management APIs come into play. The first one is the file system access API. And this API allows you to do any file management operations. For example, read, write, create a new file and also manipulate in the authorities. In this example, what we are going to do first is to show a file picker so the user can open a file. The user is going to be presented with something like this which is what you typically have with any type of application. Once they pick the file, you're going to receive a file handler and the file handler allows you to, for example, extract the content of the file, the text and use that text to show it in the screen. In this case, we are using a text area and we are adding the text to the text area to show it to the user. So that's how you basically open a file. Another typical operation is save us when the file doesn't exist. And what you can do here is to show another dialogue. We are passing here an options object which contains the extension of the file to make it easier for the user to save it. But the dialogue is going to be once again the same one that the operating system usually shows. And then you can save the file is going to be saved in the file system and that's pretty much it. And finally an operation which is kind of updating the file that's already open and exists. So you have the save operation here. What we are doing is to use also the file handler and you can get the file system writable file stream and save the contents from that text to the file, for example. There is one specific use case where this API which is the file system access API might not be of help. In that API you have like one-on-one mapping of files with your file system. So basically if you open or save or create a file the user then can go and browse that directory and they're going to actually see the files. So there is a one-on-one mapping it's not hidden from the user at all. In this other API which is the origin private file system this is different. So you have actually a private file system to the origin of your site that all your site is going to see and be able to manipulate. So in this case there is not a one-on-one mapping between the files that you use in your application and the actual file that gets stored in the file system because it doesn't matter. You only need like a virtual file system and that's how this API is going to help you to do that. What's going to happen is that you manipulate the files you can open, save, do whatever you want and when you have to see the actual files being persisted you might have a different representation. So it really depends on the actual implementation but I don't know some browsers might use a database for example to store these files, it doesn't matter. So your application is going to be able to use them as you usually want to but your users don't need to see this file necessary. So that's how the origin private file system works. I'm not going to show a code about this because we don't have much time. So if you want to know more about the origin private file system please check out this article here and you can see all the information and how you can use that API as well. And the last thing that I'm going to say before going to Andres so he can cover his part is about storage limits which is something that we usually ask about. So each browser has a different quota limit. They have implemented their own strategy for this. So Firefox allows you to store up to two gigabytes of space per origin. Safari is going to allow you to use up to one gigabyte per origin. And then for Chromium based browsers including anything that happens in ChromOS you have up to 60% of this space per origin. So it's a lot of space that you can use on your PWAs. When you reach to that limit each browser is going to do something different. So Safari is going to tell you from this point you can't store anything. So do something about the data you have or you are not going to be able to store anything else. And Firefox and Chromium have implemented at least recently used policy which means that they are going to start removing the data that you don't use very frequently to make room for the new one that you are going to add. You can implement your own strategy and the ones that are more commonly used are deleting the oldest content for example or deleting the heaviest content that you have on the cache or even letting the user think what they want to do when you reach to that amount. I think the app show team also implemented their own strategy to kind of get rid of the amount of space when they need to free up for more. So if you want to actually go deeper into any of these things we have a lot of resources about that but I think the entry point for everything is this article in web.dev which is going to take you to each specific technology. So go to web.dev storage for the web and I think that on my side that's pretty much it. Next month, we're going to cover some upcoming mechanisms like SQLite that are coming to the web as well but so far I think these are more or less the ones that are more typically used. So I'm going to stop now my part and I'm going to go to Andres so he can share app shows case. So I'm stopped now, stop sharing. Andres, feel free to start sharing your part. Okay, let me see how I share this. So the really hard part of this is finding the correct slide. One second please. No problem. Okay, here we are. Okay, I'm guessing that everyone is in my presentation right now. So welcome everyone. Today what I'm going to share with you is the unique challenge that we faced when we were trying to move away from a Chrome application to a progressive application. So first, a brief description of what App Show is. We are a digital signage product company that now we use a PWA to display content on various TVs from hospitality to healthcare to shims. Our platform is 100% one base. It has been like that for the last seven years and we have a use case where we have some really large media assets, mostly videos, either for entertainment or advertising or anything that we need to show on the screen. And that is going to be shown repeatedly over the course of the day. Those ads up on our transfer bill if we don't have any sort of cash in layer around. One of the impact that we have when we switch from the Chrome application, sorry, to the PWA, since we are web-based, is that we notice a huge introduction on the data transfer cost, 50 time less data transfer usage from other CDN to the devices. And also since we were using Chrome applications, we were really narrow down the markers that we can reach when we release these as a Chrome application, not PWA, sorry. There was a 7% increase on customer reach. We expanded our market share. So, first, when we're making this transition between the Chrome application to the PWA, we face some difficulties given some requirements and constraints. Further requirement that we have is we want it to be cash fast. If our file is on the cash, we should serve it from the cash. If it's not on the cash, everything needs to be continued working as it is. We will show them all this file and keep it around for the next time that we are going to need that file. Our companies, the backend infrastructure that are going to be shared between the Chrome application, the PWA and some other customers need to have us minimal to no changes at all. We were going to this transition from the Chrome application to the PWA and the core of our application is exactly the same. Sorry, I have someone on the door. I want to disable that. The same kind of happen. As we were talking about, then we have some constraints that we have as a challenge for this kind of transition. First, we couldn't re-cash the file for some reason that we are going to talk later. And second, there is a wide variety of available storage sizes and asset sizes. We don't have that much control of those. So given those constraints, we will require a clever and innovative solution that was not exactly matching with what we started working on, that was Warbox. We started with Warbox on our PWA when we implemented the same worker, but it's not exactly what we wanted. So we had to find some creative solution in order to sort this out. So let's get to the main issues that we faced. First, range request. I don't know if you know, but anytime that you use a video tag on your browser, the browser is probably going to do a range request. That is going to enhance the capability of the tag. You can, for instance, seek for that away in the video and it's going to just download a portion that is missing and start from there without trying to download the whole file. That is not something that we can mess with. So there is nothing that we can actually do from the browser perspective in order to disable that range request. So we might do something under our backend. Whenever you do a range request, it's perfectly normal. If you want to just send, okay, we are going to give you the whole file, 200, okay, this is a whole file, and they're also going to handle that well. But again, we didn't want to mess with our backend and our backend, the CDN that we were using by default, a response with that 206 range request, response, sorry. So given those constraints, there was a way to disable that on the CDN. We didn't want to actually do that. There was some use cases where our product will benefit from having those range requests on the first time. So we were reluctant to actually remove that. But we noticed something, experimentally we noticed that even when their request was sent from the browser to our CDN, and the CDN responded with that, okay, this is a file, 206, this is a range. We noticed that the whole file was being sent. So whenever you have a range request, you have a header that says, okay, this is from this byte to this byte. We were noticing, okay, this is the whole content length of the file. So if we are having the whole content length, while we're caching that, that is not going to work by default on Warbox. Warbox expects a 200, and that is what we're going to use. If you want to use a 206, you need to implement something else. On our case, we solved this, inspired by some give-have issue, addressing this exact same situation, by using a custom plugin for Warbox. Our custom plugin cache partial is complete. What it entails is, okay, if you got a 200 response, then the whole file just cached. If you got a 206 response status code, and everything seems to be fine, the content length of the file matches whatever you have requested, from byte zero to the content length minus one, then you actually have a 200 request. So we convert that response from the backend as a 200 response, and we send it away to Warbox, and Warbox doing their magic, is going to cache the file, and is going to be available for the processor to consume. That's coupled with a Warbox plugin, that is a Warbox range request, that actually serves the request as a range request. So if you want to have sync abilities, if you want to resume from a given point of time of a video, you are going to have that, even if the file is on the cache, and it was initially cached as a 200 response. That was one of the problems, but what happens when the storage is not enough? Because one of the main issues that we have is we have a ton of videos. We have videos for news, we have video for viral videos, we have videos that are just plain advertising from hamburgers to shin glasses. And the amount of space is quite limited on a browser. Even if it is 60% on Chrome base browsers, you are still limited, it's not infinite. Well, we could just rely on the LRU that is being implemented, not being a browser, but that is per origin. So whenever we are going to hit that bolt error that is three or whenever you don't have any more space, that's going to wipe everything. And that was not enough. We want to keep some files around that are frequently accessed. And a solution to that is use one of the many world's plugins that are available. And the main one is, okay, if we can limit this for the amount of files, or we can limit this for the file that is older in the cache, or we can just wipe everything whenever we get some of those caching errors. But that's wasn't exactly what we wanted. We wanted to, for instance, if you have a really large file, a couple of gigabytes worth of video, and you want to continue display that on the screen, you're not going to remove that. That is going to stay on the cache. But maybe there is some file that is older, some file that is not accessed for a long time. We want to keep that. In order to do that, we use another of the tools that Demian also told you about, especially with the library that converts indexeddv into promises. And with indexeddv, what we did is, okay, whenever you are going to access a file, just store the last access layer on this indexdv. And whenever we are running out of space, just go through the database, search which is the oldest file that you can get over there, and remove that. And that basically what it does is for our origin that we have only a couple of origins, it creates an LRU eviction strategy based on the access time of each individual entry, instead of wiping everything or removing something that is actually being used right now, or the oldest access entry or for the amount of entries. This is super helpful for us. And this approach enable us to ensure that the optimal usage of the cache size is going to be used on that browser, and that we are not going to remove anything from the browser that may be used later on that day, that week, or whatever. Now, going back to the impact, we did this with two remarkable achievements that just turned out. The first one is that we navigate this unique challenge of catching large video with viral storage sizes and video on five demands, without altering anything from our backend or the core application. Everything remained exactly as it was. We had a Chrome application, we just removed some portion of it. We implemented service worker with a PWA, Wordbox, some custom plugins, everything kept working. We kept our code as clean, coherent. We were crafting a precise solution for the problem at hand. Secondly, this is not only a technological success, it really translated into a real business impact by enhancing this cache capability and moving away from the Chrome environment. We unlocked this new market that was not visible before due to the operative cost of having to download these files over and over again. So we expand our reach and open new opportunities. And that's basically our story on the transition from Chrome application to progressive web application. But to you, Demi. Yeah, I think Rich, do you want to continue? Yep, I'll take it from here. Thank you. I think there is a recap at the end, Rich. Yeah, thank you. I can go over that very quickly. Yeah, I'll go ahead and share my screen. And by the way, thanks, Andres, for the whole presentation. We are going to be sharing the materials afterwards so you can take a closer look at every part. And I have a few questions to ask you as well. So yeah, I just wanted to just make a summary for those that are willing to use any of these technologies. So at the beginning, we saw some of the storage mechanisms that are available. Some of them are still useful, but they're quite limited in terms of the amount of data that you can use. And some of them like WebSQL or the application cache are either deprecated or not longer working. We also saw that most of these are synchronous and they work only on the page. So you can't really move them to a secondary thread and then take advantage of that. The cache API and index DB are the ones that are most frequently used. And I would say cache API mostly for things like static resources. And especially for offline when you are using a service worker and things like that and index DB for almost everything else because it has a lot more features to manage the data. And finally, we have seen some things about the file system. So how you can directly manipulate files in your file system and another variation of that which is the origin private file system which lets you have your private virtual file system for your app that's not necessarily user-visible. And I think with these options we have covered most of the different parts. And finally, we have Andres presenting up show case how they were able to cache multiple megabytes of gigabytes, sorry, of video using the WorldBox library and service worker. So I think that's pretty much it, Rich, on my side. And I don't know if we have some time for the Q&A. Yeah, we do. We do. All right, thank you, Damien. Please remember to join us and register for our next live session on September 27th. That one will be about APIs for web apps. Registration is now open so you can go ahead and register.