 Okay. Good morning guys. Hope you can hear me at the back. Cool. So, most apps today have to communicate with the server, right? To pass data back and forth. So, in today's talk, in the next 40 minutes, basically, we'll figure out how that communication layer could be optimized. A brief about myself. I work at height doing all things Android. Let's start the discussion by talking a bit about height, how the journey has been. We launched in December 2012 and recently crossed the 35 million user mark. 90% of our user base is in India. That has governed a lot of stuff that we do. In terms of growth, x-axis is the number of days and y is the users in millions. We are experiencing a lot of growth reflected in this chart. Now, this is interesting. More than 10 billion messages are exchanged on height every month. So, given all of this background, what does building for India mean technically? I mean, there are a lot of challenges which are business oriented while developing an app for India, like localizing the content and forming telecom partnerships. But in this talk, we're just going to talk about the technical challenges. See, if I were to pick two things that as an app developer, you should care about. If you're targeting the whole of India, it would be these two things. Network optimization and getting to low-end devices. If you remember from yesterday's talk by Gulzar, he mentioned that how a lot of network connectivity in India is flaky and most people are not on Wi-Fi and 3G, the way we would like to believe. We at Hike experience all of this while building, and we continuously optimized our network communication layer to cater to that. So, first things first. Most people in India use Sashay data packs. Sashay data pack is something you go to a store, buy a 50 MB pack, and charge your data connection with that. So, how many of you here use Sashay data packs? Only one. So, remember, if you are building for India, this is not your target audience. The target audience is something who rely on Sashay data packs. See, a lot of people are still experimenting with data. So, people going to colleges, people in non-metals, they're still experimenting with data, figuring out if it actually is worth their money. So, the first step of experimenting is with Sashay data packs. It roughly costs around rupees 3 per MB. So, imagine you're downloading a 20 MB file, or a 2 MB file, let's say. It costs around rupees 6, more than a cost of printed paper. So, with this in mind, any layer that you implement for data thinking has to optimize for bandwidth. It should not happen that the user's installer app does not use it and figure out that 20 rupees have been deducted, just because you were syncing some data back and forth. So, when the first step obviously is to design a... So, think of a client-server protocol. While developing for Hike, we had these three goals in mind. A communication layer had to be persistent. So, this is a specific requirement for messaging apps, per se. By persistent, we mean when a message is sent, it should be received in near real-time. There should be no gap between establishing a connection and then delivering a message. So, a requirement you need to messaging apps, basically. The second one talked yesterday as well in the keynote. We have to take care of flaky network condition. This is by far the one thing that you should care about when developing for India. And obviously, you have to consume lesser bandwidth. Whatever protocol you choose, think about is it optimized for low bandwidth conditions or not. With all of this in mind, we zeroed in on something called an MQTT protocol. So, it's a protocol for broadcasting and listening. A good thing about this protocol is that its header to message ratio is very, very less. The amount of overhead in terms of bytes used, which is non-message, non-payload, is very, very less. Support something called a public subscribe mechanism wherein each node can subscribe to particular topics of interest and receive messages which are only pertaining to those messages, only pertaining to those topics. And the last one is pretty important. It supports quality of service levels. So, now it recognizes that not all messages are important. So, why to jam the traffic on the network by receiving acknowledgments for the all kinds of messages flowing in the system? We'll talk about this QS levels in a few minutes. So, typical MQTT topology consists of a server and a client component. There could be multiple clients, multiple servers as well. And the protocol works by exchanging a series of control packets. We'll see how these control packets look. This is a typical control packet, having a fixed header, mostly two bytes. And then there are variable headers and payloads. So, these are the 16 kinds of control packets that MQTT supports. There's a connect control packet which the client initiates to establish a connection with the server. There's a corresponding CONAC, which is basically an acknowledgement for the connect control packet. Then this one is pretty important, the publish. This is by far the one that you will use for communicating with the server. So, any payload that you have actually travels on this publish control packet. Corresponding to a publish control packet, there's an acknowledgement. And these series of acknowledgements basically are governed by the QOS, the quality of service level that you sent with the published message. Then down here, there's a subscribe and sub-ac. So, the clients can subscribe to particular topics of interest and receive a corresponding acknowledgement. Then there's unsubscribe. The number 12 here, ping rack, it is pretty important. So, at least from a point of having a persistent connection. So, we'll talk about this in the next few slides, the ping rack and the response. So, this is like a typical flow for a MQTT establish, connecting a client to a server using the MQTT protocol. The client initiates a connection with a connect packet, receives a CONAC, fires a subscribe to a particular topic of interest, receives an acknowledgement back, publishes messages, receives an acknowledgement for those messages, and then finally disconnects. So, this is a typical format that a lot of MQTT-aware systems follow. But we sort of delve into this and figure out where are the bytes that we can still save over this format. We said, let's do away with the subscribe and sub-app and do an auto-subscribe at the server level itself. I mean, given the scale of 35 million users, it would seem like, you know, just avoiding two packets. But at the scale of 35 million users, you will figure out, it amounts to a huge network traffic that we have avoided. So, the connect request itself makes the server do an auto-subscribe for that client, coming to the QoS levels. So, like I said, MQTT recognizes the fact that not all messages flowing in the system are of equal importance, right? So, there are some messages for which you can use a fire and forget policy, like at most ones. You fire the message and you don't care about whether it's received at the receiving end or not. A typical case where you would use this is, let's say in the chat app, you have the typing notification. The friend is typing something and you receive what is typing notification. So, that is something you can do with the QoS0 because that notification itself is of lesser importance because multiple such notifications are coming back. Then the QoS1, this is by far the one that we use at Hike as well. It guarantees that the message you publish would be received at least once by the receiving end. In our case, it would be the server. It does not guarantee that it won't be received multiple times at least once. Now, there is a QoS2, which is exactly once by a series of acknowledgments flowing between two ends. The QoS2 guarantees that the message is delivered only once. The overhead is that a lot of acknowledgments are flowing back and forth in the system which consumes a lot bandwidth for the user. So, we decided to zero in on QoS1 so that messages are delivered at least once and build logic at the client as well as the server end to flush out the duplicates. So, over and above the MQTT layer, we implemented active and passive term-outs. What happens is to maintain a persistent connection, the client on the device basically would send a Ping Rack message, the number 12 control packet that we have seen. Let's say every 10 minutes, a Ping Rack message is sent if there are obviously no active messages flowing in the system. Now, for the server to still maintain a connection, it will figure out if a Ping Rack message is not received within those X minutes, within those 10 minutes, it will disconnect and fire a GCM push to the client device to wake it up. So, we use GCM push for a fallback mechanism because the network connectivity is flaky. We don't know whether the system has used a half-open socket connection thing. So, we utilize the GCM push which obviously Google and other Android services provide to wake up the clients. This is for passive state when no active messages are being exchanged in the system. When there are active messages, the client is in an active state. I publish a message and because this is like a real-time chat application, we use an aggressive time-out. If within one minute, we don't receive an acknowledgement for a message published, we do a time-out and reconnect at the client end. So, a distribution between passive and active time-outs. So, coming back to the, going forward to the next topic, which is file transfer optimizations. A lot of apps knowing and unknowingly do a lot of file transfers. So, you have a list view containing images, you will be doing a lot of file transfers. Just to give you a context, more than three billion stickers are exchanged every month on Hike Network. So, these stickers are nothing but image files for us internally. The first rule to do file transfer optimization is to avoid when you can. Avoid the transfer at all. The easiest thing, don't load images on on scroll. The user is just scrolling on a device. Just stop the loading of images at that time. He may not be interested on all those views. It's a very simple thing, but a lot of times we just miss it. You can also implement something called an MD5 checksum. So, this is pretty interesting. At the server end, the server maintains a table containing the MD5 of each file being transmitted over the network and a corresponding key. Now, when a client has to transfer a file, it will first, at its own end, calculate the MD5 of the file locally. No server transaction required. Then transfer the MD5 to the server. The server checks in its table, whether this file was anytime recently transmitted over this network. If so, it returns a corresponding key. No file transaction has happened yet. If a key is successfully received by the client, the client fires a published control packet with the key, not the actual file. And the server then processes that message and maybe sends it across to the destination network. So, what this helps us, this MD5 checksum, is to avoid the file transfer at all. Now, in its plain implementation, the table at the server could grow very, very large. I mean, like, three billion messages, three billion stickers are exchanged on the Hike network. So, with each file transfer, we implemented a TTL, a time to live. And whenever a file is reshared again, or forwarded, we increase the TTL. And for files, which are no longer shared, the TTL times out, and those are removed from the table. So, this is pretty helpful for images or messages, basically, that go viral very, very quickly. You can also use a unique ID based mechanism to avoid the file transfer. So, for stickers internally, we have a mapping for each sticker with a corresponding unique ID. And when a client one has to transfer a sticker to client two, it does not transfer the whole file. It just publishes a message with a unique ID to the server. The server passes back of hours that message directly to the client two. Now, client two checks whether there's a corresponding sticker in its own local table, right? In its own local table, and if it is present, it uses that to display to the user. No transfer of sticker has happened in the whole process. If the client, the receiving client does not have the corresponding entry in its table, it will request the server for the file corresponding to its own, for the sticker ID, receive it and display it to the user. And correspondingly, it will also make an entry to its local table. Always transfer based on the destination DPI. So, example, a server is transferring a sticker to a device which is MDPI. No need to send an access DPI device, access DPI image to the device, only to further down sample it to the MDPI resolution. So, all of this means that you'll have to write more code both at the client as well as the server end. But eventually, it saves a lot of bandwidth that is flowing in the system, that is being utilized by the clients. So, in yesterday's keynote, Gulzar talked about how flaky the networks are and whether there are apps which take care of the type of network the user is connected to, whether it's 2G, 3G and Wi-Fi. We at Hike do take care of that. In fact, we have optimized for such networks. So, imagine transferring a 100 MB file which is something that Hike supports. Imagine transferring a 100 MB file to your friend. Now, what happens if the connection, if like 70 MB has already traveled and then the connection breaks? You will have to retransmit the whole file again. It's a huge, huge waste of the bandwidth. So, we divide each file into chunks, various chunks and then transfer these chunks individually to the network. Now, both the client and the server obviously have to maintain a state of the chunks which chunks are received which are not like more, still more code, but eventually if a chunk is lost, we transmit only that chunk. No need to transmit the whole file again. And the interesting part is that even these chunk sizes, these vary based on the kind of network that the device is connected to, right? If it's a 2G network, the chunk sizes are pretty, pretty small, like 32 KB. If the client is connected to a Wi-Fi, we increase the chunk sizes to around one MB. We also experimented with multi-threaded download. So, if one thread is still waiting for an acknowledgement, why not start another thread in parallel to download the remaining part of a file? We are still not done in terms of optimizing. We keep playing with the chunk sizes and now we are experimenting with varying the chunk size at runtime. So, based on the actual speed, instead of saying that we statically determined whether it's a 2G or 3G Wi-Fi, we actually change the chunk size at runtime. So, even if you transfer Wi-Fi, but the speed or there's a packet losses happening in the network, we'll trim down the chunk size. The last part is basically feature optimizations. App decisions at Hype are also based on how to optimize for the network. Can we design features that actually consume lesser bandwidth? As one of the keynotes mentioned yesterday, a lot of users in India switch on and off their data packs. So, you'll go to a college and maybe if there's a break, you will switch on your network, check your messages, and then turn it off. And maybe in the unit, you again turn it on. So, in a chat application like ours, if a lot of your friends have sent you messages during the time your data was off, those keep getting accumulated at the server end. Now, when you come online, there's an option for us to do nothing special, just transmit those messages as they would have been received, like individual messages, which will mean that these messages travel with their own headers. Each message travels with its own header and have a corresponding acknowledgement for that particular message. We implemented bulk packets, wherein all these messages are actually clipped together and sent as one message to the receiving device. Again, more code at both the server and the client end, but optimizes optimization for the network. Further, you can even Gzip, which we do, you can Gzip the packet itself, if it's more than a certain size, that's a 10 KB or something, and then transmit the bulk packet. So, in a chat application like ours, if you have added another user as a favorite, you can see his last time, like when was this user online last time. A simple way to implement such a stuff is to sort of broadcast each user's connected time across the whole network. We moved into a subscription mechanism rather than a broadcast mechanism, wherein whenever you start a chat, at that time itself will fetch from the user what the online or offline status was. So that helps optimize a lot of packets that would otherwise have flown in the system. So in summary, network optimizing should be your top priority if you're building for India. This is not the target audience, or this is not the only target audience that you should be building for. Think about people who are still experimenting with data. Think about how the network connectivity is flaky in India and optimize for that. Choose a bandwidth optimized network protocol. We use MQTT, Facebook Messenger also uses that. Think about how you can optimize the file transfers. Maybe use MD5 or a unique ID based stuff. Always take care of the resolution that a client device is on. Think of the features that you have implemented and figure out whether optimizations are still possible. I am open to questions now. You spoke about optimizing the image size to reflect the DPI of the target device. Now I have a question here. So I've seen a lot of Pujaris use something like WhatsApp to transmit the text written on the book from one side to the other. So in such a scenario, what would you do? Yeah, so this is interesting. So there are two parts to it. One is an image that you already know of, like stickers in our case, which you already know of, we have already re-sampled to a destination device. Therein, obviously we can use a destination DPI waste. But for images that are user-generated, for example, the Pujari case, we have given an option to the user itself, a user himself or herself, to whether they want to downscale the image or transmit the normal file itself. So because at the time of transmission, the sending guy will not know what the destination DPI of the receiving end is. So in that case, we have given the option to the user. A lot of users now understand that transferring huge files will consume a lot of bandwidth and basically transfer a lot of money for them. So when a pop-up comes up, whether you want to transmit the file in its original resolution versus trim it down, they typically choose a trim down version. I think one of the cases where you can help the user make the decision is to show the files size actually for each of these resolutions. You can say original size would actually be 1MB, trim down version is only 40 KB, and maybe tell them that it would be a faster transmission if you choose a smaller one. But people obviously who are transmitting the whole want to send actually the original image. Maybe they want to, the receiving guy wants to print it somewhere. They will prefer the original size itself. And just letting the user know what all these options are pretty much will help your case. Hi. Hi Kira. Yeah, sorry. Thank you, it was a very good talk. I have two questions. Firstly, as you said that network conditions are very flaky. We as users also face it all the time, even however good operator or a data plan you have. So I wanted to know that like, do you have any statistics or something that you can share? How many times does the client the app actually gets through and how many times do failures occur or something like that? So we do have statistics but I don't remember them offline, like off-hand right now. But maybe you can connect to us in the breakout area and sort of we can dwell into it and figure that out. But on a general level, it is pretty, pretty common, more common than we can sort of think of. When we develop in our sort of environment. So Rajat is basically a CTO from Hike. So your question was like how many times we end up making these connections there, correct? So what we have seen is even over a very large user base, we average out around 30 plus connect attempts every day. So that's how many times a person moves in or out of a network. And that's to average out over such a large base. There are outliers which goes like way beyond what you can imagine that like. Hey, follow up from the same question here. So you said 30 reconnect attempts, right? So could you walk through what it takes to sort of reconnect? Is it the same process or the connection? So we talked about passive timeouts and active timeouts. So imagine you are in a half socket connection or the network connection has gone. The server determines that this, the client is no longer reachable and it receives a message for that user. So it will send a GCM push. So there's no way for the server to directly reach the client now. So you bypass the MQTT protocol itself. You send a GCM push, the client awakens and reestablishes the connection. The client understands that whenever you see the GCM push, it means the server no longer has a connection open for me. So let me disconnect and reconnect again. Yeah, so it's a, so it actually will depend on the kind of app you are building. So if you're building a messaging app, you'll actually want a real-time communication to happen. So the other way for this, for the way we could implement it is there's no active connection, right? And anytime a message is received, we'll send a GCM push and the connection happens. But that would mean we lost precious seconds. So if you send to your friend nearby, you know, send a message and he'll receive after five seconds, you'll say this app does not work. So for a messaging app like us, we have to have a persistent connection and that's why there's this requirement. But in a general, if there's no real-time requirement, you may totally avoid the plain rack control packets. Excuse me. Right now I want to thank you for this. Actually, I'm not, yo. I want to thank you about this. But I just want to know how much of your user interface is hybrid? And how you are taking care of multiple platforms? I think it's a great question. So this is something that every app developer sort of struggles here. Our apps are totally native right now because we think that provides the best user experience. If we were to develop a hybrid app, it would obviously mean lesser time to develop, but it will also take away a lot of unique experience that these native sort of provides. So right now our apps are native. We are still experimenting with something called a J2OBJC, if you have heard about it. So it's an open source framework that Google had released. So we are experimenting. We are figuring out, can we share that model part at least, not the view part, but the model part across Android and iOS. That should sort of trim down the development time. But as of now, all our platforms like Windows, Mobile, Windows, iOS and Android are totally native. So do you prefer hybrid? So as I said, until the technology comes up to a level wherein we can effectively give a user experience. Actually my basic idea is to know whether hybrid applications are more better than native or whether native applications are more better than hybrid. So it all goes- Because if I am developing a hybrid chat messenger like Hike, okay. So it will cope up with all the platforms on a single compile. So my basic idea is this. So I understand that. So from a developer's point of view, obviously you will not want to write one code base, right? But if you think of the end user experience, if you develop an HTML, you can still make out, right? Is this a web view or not? We certainly use web views in our app wherein we think that the content is very, very dynamic. For example, the rewards. We sort of provide rewards to the user to invite others and stuff like that. So all those pages are still web views because those things are not part of the code missing app. Having said that, we are experimenting on that as well. We're figuring out where is the HTML technology. Can we utilize something there to build things in future? Rajat, can you answer that? What is the backend technology that we use as a server? Yes. There are array of technologies. So like what we've done is our front, like the place where your connection terminates is based on HAProxy. So that's our load balancing layer there. Behind that, as Parish mentioned that we use MQTT as a protocol. So we looked at implementations of MQTT server, could not find one which meets our requirements and can scale up to us. So we wrote our own MQTT server. It's a purely Java-based server there. We use JMS, Hornet Queue Implementation for inter-machine communication. For storage, we use MongoDB at some places, batch-based, some places SQL as well, MySQL as well. And we use Redis a lot for our caching layers there. To the extent that our Redis clusters, some of them run at 250,000 operations per second. So a lot of open source. So like as any other startup, we are- You have never adopted for Erlang. So see, it was again like Erlang we could have used for the connection termination. And I know that we could have written less code to achieve that. It was a choice based on what experience was available in terms of our developer talent here. And we said that we'll fix it and we'll go with what we have, optimize it to the best. Currently, each of our machines are 32 GB box, holds up to 700,000 connections. So yes, like our targets are still high to say that how do we take it to million or million plus connections? Erlang could have given some leeway there, but that was like how the development experience was available there. So a choice was governed by that as well. Thank you. Hello sir. My question here is, what is the privacy level of FIKE? Basically. So we started with like, so if you see the initial versions of FIKE, we were actually the first time we said that we'll honor and we'll give you way more privacy control than any other app. So WhatsApp has that problem, has earlier the problem of saying that anybody can see your last scene. So Hike had a concept of that, yes, you have people talking to you on Hike, but then you have friends to whom you want to restrict that only these people should be able to see your last scene. That was our effort on privacy. We went a level ahead to say that sometimes, like again, when we say building for India, we actually see what the circumstances are. So in India, we share mobiles, correct? And there is a lot of communication that you will not like anyone else to see. So we came up with this concept of stealth chats or hidden chats there. You can easily mark a chat as hidden and as soon as an app goes into background or you can just at a click of it hide your chats there. So Hike remains the same minus few of your personal chats there. So we have given kind of options to say that what kind of privacy you want to control. Whom do you want to show your picture? The display pick to your friends or no one or everyone. So we have given those controls and our default is like go for more privacy than less. So by default, your last scene only goes to your favorites, the people who have you exclusively given access to. So by giving, by marking up a person as friend, you are telling me that, okay, share my last scene with it, share my profile pic with it. But by default, we are saying that we'll not go with everyone but we'll go with this thing. And most of the other apps go the reverse way to say broadcast to everyone and then you have to restrict stop, stop, stop, stop. We say the reverse way there. Hi. Yeah. So one more question regarding MQTT. Yeah. So as I understand it works on top of the TCP IP stack. Yeah. So it works on TCP. So obviously there is some bit of reliability built into it, right? So I don't know much about MQTT. So how is it actually, you know, what is the control flow in which it guarantees liability and what is your experience in implementing it on Android? Is it heavy weight? And do you have to import any other libraries or how is it? Great. Great question. So first thing first, MQTT sits above the TCP layer, right? It's an application protocol. It does not care whether like TCP or IP is used underneath. Obviously TCP guarantees that the individual packets are actually being received by the other end. But what in case the application layer guarantees that even if the data is sort of switched off or the other end is not reachable, I'll retry later on. So that is the reason behind QoS1 and QoS2. So even though TCP is providing that reliability, but at the application layer also, we need to have these controls which MQTT provides. At the Android level, we implemented MQTT protocol using a library from Eclipse called PAHO. And we customized this library to suit our needs in terms of the buffers and everything that we should be using. But we went ahead with PAHO. There are no power consumption issues or something like that. So this is the best we can get. So if you use some other protocol like XMPP or something, they have a lot of overhead. MQTT was specifically designed for low bandwidth consumption and flaky networks. So originally it was developed for IoT, like Internet of Things stuff. So there would be sensors who would be using MQTT inside them and transferring that thing. But now it has come over to these mobile devices because we face the same issues. Right, right. Thank you. Testing strategy. So we have developed automation suites for each of these platforms. And obviously we test manually. But more and more of our test cases are now done through automated test scripts. For example, whenever a build is created, whenever a build is created, an automation test will run over all that code that has been checked in and figure out whether something has broken or not. For example, a lot of users, sometimes in India, don't upgrade. So we have to figure out what the upgrade path is, whether there are any kind of issues when users from previous version, even from the version one, if they are upgrading, are there any issues they are facing? So all of that is actually being run through automated tests because at this scale, the manual testing will fail. There are so many cases that you have to handle. Right now, we have separate build systems as well as separate test suites. But there are efforts going, even in the testing automation stuff, to unite all these and figure out where can we utilize from each of these. And not only that, we have not only feature automation, but automation for battery consumption, all those things are also taken care. And if we have Nitesh here, so he's reading our testing automation framework. So you can catch all of us in the breakout area and maybe Chad will sort of give you the details on all of these things individually, like go deeper in all of these things as well. Hello. Yeah. Yeah. So are you guys using, on top of MQTT, are you guys using any kind of encryption or is it done in plain text? So MQTT is sort of unaware of any encryption protocol that you can use. We have provided an encryption option to our users using SSL. So if they want to encrypt their chats or anything, so they can use this SSL option by going into some menu and sort of encrypt the conversion that are happening. So it's done before giving it to MQTT, the data package. After. So there's IP, TCP, and then SSL, and then MQTT. All right, thank you. Folks, please proceed to the speaker lounge or download the talk funnel app to ask some more question and answers to Parish.