 Good morning everyone. I am really glad to be here. My name is Surya Kanodia and I am working in the cross-platform team at Flipkart. Now, we were one of the earliest ones to ship out a very complex feature on ReagNative at the scale that we did. In this process, we learned and we had to face and solve a lot of issues which are probably relevant across the world but it is really important in the context of India where sending out large amount of data over the internet especially to mobile devices is not recommended. Today, I am going to talk about one of these issues but before talking about instant update patches, let me give you a brief background. We started with ReagNative to leverage the cross-platform capabilities that it offers. You also wanted to have the capability to skip App Store and Play Store release cycles and deliver over-the-air updates to millions of our users. Now for this, there was already a lot of existing over-the-air update or OTA solutions out there but all of these solutions, they had certain issues which were really critical for us. The first issue was that the updates delivered by these OTA solutions, they were enormous in size. We generally ship out an update every week so we cannot afford that our users download an entirely new bundle every week. Secondly, we have multiple teams working on ReagNative at Flipkart. Now, each team likes to maintain their own bundle. In web, you have the concept of vendor files so you can ship your common libraries, common code, common framework as a part of the vendor file and this gets reused across all your single-page applications. ReagNative, it does not provide the support for vendor files. It needs just one single bundle so we had to find a way to share code. Now these issues, they prevented the updates delivered to our apps from being instant. So it did not make any sense for us that we have shipped out an update and our users, they use a bundle which is fetched from the cache while a new one gets silently downloaded in the background. Okay, so your first instinct here would be to use a standard text-diffing algorithm on the server side to generate your update patches. Now, at the scale of millions of users, when you need to support app versions that you have released two years ago and maintain update patches for more than 200 bundles over the last two years, for just one native platform for just one bundle, the solution becomes really complex. What we did was, we just offloaded the text-diffing part to the client side and this reduced the overall complexity of the solution drastically. We built up such a system and we call it the dynamic update service or DUS. Now before we started working on DUS, there was this backend service which was already in place and it was used throughout the apps. This is a very simple to build and scalable REST service. It supports the basic create read, update and delete operations. It was backed by a key value storage pair and it had support for multiple keys. So if you send in a list of keys to this service, the service would return the corresponding values. In the context of DUS, we call keys the hashes and values as the documents. This service, it also provided a very nice thing. It supported querying based on parameters like app versions. So if you send in your app version along with the query, you would get the documents corresponding to your app version only. This enabled us to send out different updates for different app versions. Before we started working on DUS, we had a very simple goal. If our users, they already have a react native bundle downloaded and we wanted to ship out R changes. R users, they should download just the changes and not the entire bundle itself. For this, we had to understand how the bundles were created in the first place and we started playing around with the react native packages. So if you run the react native bundle command in your repository, it will traverse through all the JavaScript files in your repository and it will create one single unified bundle. Now we ran it in our repository and then we made small changes to just one of the files. We ran the command again. The second bundle was very different from the first one. We realized that there was some sort of obfuscation going on. So we made some modification to this obfuscation, which I'll talk about in a while. After making the modification, we repeated the entire process. Only this time, just one of the lines had changed. From this, we concluded that one line in the minified bundle corresponds to just one module or component in your react native repository. So each module has its own line in the minified bundle. This was a very significant information and this is what our solution is based on. Let me tell you what changes we made to the obfuscation. So basically whenever the react native package encounters a new module, it assigns an ID to it in a little unpredictable way. We replaced these random IDs with a hash of the file path where these modules were present in the first place. After making this change, whenever we ran the react native bundle command on the same repository, we always got the same output. So we realized that diffing was possible and it was possible to generate update patches. Now it's all about generating them and sending it out to our users. So let's talk about how we generate the update patch. Our update patch is divided into two parts. The first and the most important one we call the update graph. The second one is a simple network call where you send in a list of keys to your API service and you get in the corresponding values. Let's talk about how we generate the update graph. So you just iterate over the bundle and you split each line into a chunk indirectly. You're splitting each module each component into a chunk and then you generate a hash of the minified code. The hash becomes the key for the key value storage pair and the minified code becomes the document. You just upload all of this to your back end service. So your update draft would look something like this. It would have the bundle names and the list of corresponding chunks associated with those bundles. So if there is a bundle called product list, you can see the chunks that are associated with the bundle product list. You just download them, you combine them and your bundle would be ready. This is a client side diffing solution. So let's talk about what happens at the client end and how do we actually apply this update patch? So during during each app launch, you get the latest update graph in case of flaky networks. You just get the update graph from the cache itself. You iterate over the update graph. You fetch all the chunks mentioned in the update graph and you combine them according to it. And that is it. Your bundles here are ready. Let's talk about how it works in the context of diffing. So let's say it's the first app launch, your cache would be empty. You download the update graph, you iterate over it and you would need to download all the chunks. Once you have all the chunks, you insert them in your cache, you combine them and your bundles are ready. During the next app launch, let's say if an update is available, you download the new update graph, you iterate over it and you would try to fetch all the chunks mentioned in the new update graph. Only this time most of the chunks would already be there in your cache. Only the files which have changed would generate new chunks. So you would just need to download the new chunks. You would again combine them according to the update graph and your new bundles here are ready. So you can see you downloaded just the files which have changed just the new chunks, the files which did not change did not generate new chunks and it was already there in your cache. So this is how diffing was solved here. One more problem that we wanted to solve was that we wanted to share code across the bundles. Again, you have to remember that each file here is a chunk. So let me take the example of an update graph on the right side. Let's say does wants to create a bundle called product list for this does needs hashes 1, 2, 3, 4 and 5 does goes at downloads them combines them and the bundle product list is ready. Now does wants to create a bundle called product description for this does needs hashes 1, 2, 3 and 4 now hashes 1, 5, 1, 6, 3 and 4 now hashes 1, 3 and 4 are already there in your cache as a part of the call for product list. You just need to download the hash 6 the file specific to product description the chunks specific to product description you would combine them again once you have all the hashes and your bundle product description is ready. So you can see here again, you did not download the common chunks hashes 1, 3 and 4 you did not download the files the common files here this is how we avoided redundant download of code which is shared across the bundles now all of this brought down the size of our updates by a huge margin but we were still not happy. We wanted to optimize even more. So let me share some of the optimizations that we did. The first one was that we G zipped our API responses. I'm sure most of you do this already but I'm just mentioning this because it gave us a huge win for a very small change these brought down the size of our updates by almost 70% the second is that we maintained versions of update draft at the rest service. So basically if the client already has the latest update rough and its version number matches with the one that we have at our service the service always returns a HTTP code of 204. So you don't need to download the entire update draft on every app launch you download it only if something has changed. Now as we kept on adding more react native bundles to our update draft we realized that the size of our update draft itself so held up significantly and this was due to the 32 character hashes that we generated at the time of update draft creation. So what we did was we just mapped these 32 character hashes to smaller unique keys and we maintained a list of these keys in a database at our end. So every time the update draft generator would require a new hash it would generate the hash of the minified code and then we would assign it a smaller key which would not be present in the list assuring that the key is always unique and much smaller. This optimization brought down the size of our update graph by around 90% during the first app launch you would need to download all the chunks before you can even render anything and we had to solve for this. So what we did was we just ship the key value storage as a part of the APK file itself. So in the first app launch you would already have most of the chunks in your cache only if something has changed would you need to download the remaining few chunks and then again you would be able to combine them and you would be able to create your bundles. Okay, so let's summarize. Thus gave us the capability to deploy new features on the fly and enabled us to have a web like agility in web whenever you want to deploy a new feature you just deploy it against your corresponding route and you make the route discoverable in your page. It's the exact same thing here. You deploy you add your new bundle to the update graph. You ship your update graph out now if your user wants to go to a news on your new feature you just open the new activity and this new activity would pass in your new bundle name to dust dust would return the corresponding bundle and that is it your new feature would be able to render. Please use this feature judiciously second dust brought down the size of our updates by around 95% from around 700 KB to an average of less than 40 KB. It also saved us more than 500 KB of comment code per bundle as so we have four bundles in production and this brought down the download size from around 2.8 MB to 1300 KB for the first app launch and this is obviously non-jizzy but for incremental downloads when you have when you are sharing common codes and you are downloading just that if the math here really really gets crazy. So this is how you build an over the air update system for react native with that. Thank you. My name is Surya Kanodi and that is my Twitter handle. Anyone has questions. Please raise your hand. I see some hands in the back there. If you prefer not to raise your hand, you can tweet at react foo and we'll get your questions more anonymously. They cannot hear you. Hello can you hear me now? Yes. We are looking to open source it as in we have already started working on it but there is still some way to go as in we should be open sourcing it in some time. Yes. No, I don't think we'll be providing us back in service to support a third party apps, but we would be providing the code to host it on your own service box. That is the idea. Yes. Hi. Hi. So my question was with older apps because you're sending so many tips won't the size of the app keep on increasing at the size of the APK file. No, the like this will be part of the data of the app. Right? Yes. You are deaf. So won't the size keep on increasing? No. So basically every time a new update graph is downloaded, you know which are the chunks that are already that are actually going to be used to just clear out the remaining chunks. So your DB it's almost always on a fixed size. Are you chunking line wise or like a like file model wise? We are chunking it line we are chunking it line wise in the minified bundle, but indirectly you are actually chunking it module wise because that's what the react native package is for you. My audible. Yeah. Can you hear me? Yes. So I just want to ask how often do you probe key is a new version available? Is it on every app foreground event or how do you type that or something do you control that also? We do it on every app launch and we also do it whenever the app goes to background and resumes and we have certain trigger points, but the most important one is usually the app launch and it takes only around 25 milliseconds to construct the new bundles on the client side. So I think it's good enough is Flipkart shifted to react native. The app is now react native with it's not a native app not completely on react native the listing page the search page it's on react native and we are adding new features on react native as in whatever new features it's that get shipped now it gets shipped on react native. So how do you do that you have some pages in react native and some in in the native app? I'm not very familiar with react native if you can take a minute out that is a whole new topic itself like the integration of a react native app and native app and making it a hybrid app. It's a little difficult. Yes. Hello. Yes, I can hear you bundling the assets. I'm sorry what am I audible? Yeah, are you bundling the assets assets assets? No, we are not bundling the assets. We already have a very nice system to download assets from its case and everything so we never really needed assets as a part of this react native frequently releasing the latest versions right in a month you will be getting a new version. So how you are handling that actually so we don't upgrade our react native code that frequently whenever as in whenever we do we will make changes accordingly as far as I know there have been no breaking changes and obviously if it's open source I would hope that the community helps me make the changes and like keep up you mentioned that you replace the hash keys with unique keys. Yes, why don't we use directly the unique keys? So if we directly use the unique keys it would lead to hash coalition, right? We need to create a unique hash every time. So for that you just use a standard hashing algorithm which avoids collision now to reduce the size. You just check if a smaller key is already being used. If it's not being used you assign it and you insert it in your list so it avoids collision. What is the like? What are the future optimizations that you're going to make into it? What are the future optimizations that you're going to make into your system? So this already brought down the size of updates for around like 40 KB for now I think we are more than satisfied from 700 KB to 40 KB is good progress. I believe maybe if we think of much more if we think of more optimizations we'll go ahead and do it right now. I don't think we will we don't require it actually. Thank you, studio. We've run out of time for questions but don't worry because it's a break and also during the break downstairs in the banquet hall there's a round table OTR so you can go and talk to Surya and Narendra. No, I said it wrong. I said it right. Oh my God. I got it right.