 I am Harshad and today I am going to talk about designing XMPP for mobile as many of you may have used XMPP for chat apps or any real-time communication like cab apps or anything. So yeah, let me just give you a question. So currently I am working on draws. So it's a search engine where I am actually, just a minute, yeah. So currently I am working on draws. It's a search engine. So where I am actually indexing all the users, bots and you can talk to them in real time. Previously I was at Amazon and before that I was at Kiwi mostly working on Android and some web related stuff. So in this talk we are going to look at ways how we can optimize XMPP, how we can optimize it for 2G. Mostly there are timeout at different stages such as when you connect to your XMPP server it times out very frequently. So we are going to fix that. We chose XMPP because of few reasons. It's still used by Gtalk, Nimbus, it's used by GCM. Hangouts video chat uses it. I mean I have added links to all code and relevant document everywhere in the presentation. So and the reason we chose XMPP was that we wanted to do many things like we wanted to have presence so that if user is online offline or if to get the chat states like if the user is typing or not. So XMPP have an XCP or a module for almost everything. Something for pub subsystems, VoIP, Hangouts is using it and then for file transfer, we are using it for file transfer. For gaming, Chess Park uses it for real time chess moves, sending notification. So one of the best parts is since it contains module for almost everything like there are 350 modules overall. So it's very modular like you can just pick up whatever you want and you can leave all the other things. One more good thing about XMPP is that if you are using a normal long live service, anything, MQTT, WebSocket or any other protocol, I just know these three. So usually all these protocols works by maintaining a duplex connection from server to client and then if you want to send a packet from client A to client B then you send that packet from client A to server stream that packet and then that's our send set packet server outside packet from its stream to client B. So this is how most of these things work. But part about XMPP it's very modular like if you have two clients on different servers, they can still talk. Your server maintains a connection in between them. Since XMPP is so well defined like you know this is going to be the packet, you can always connect your client A to server. I mean you can always send your, from your client to a Gtalk client the packet. So many of you, I mean many of us can use it for growth hacking like Slack provides XMPP external interface. So if you want to write something against Slack you can do that. Gtalk provides, you just need to know the client's ID. That's it. So here I'm just going to talk about the issues with XMPP. You can see the packets, message, presence, IQ. These are all XMLs. So it's streaming XML. The packets are huge. You can see like the packet on the top is around 300 bytes. So it and compared to some other protocols it's really verbose. But there are ways to fix it which we are going to talk of course. Also we opened the Duplex Stream. Before opening the stream we need to do something called session establishment. This is huge. Kind of you are getting your friend list before that which it can be like 1000 friends at once. Then you are getting your presence information. Probably 500, 500 friends are online. You are getting all those status who are online, who are changing status. You are doing authentication. It's huge. It frequently times out on 2G. So this is something we are going to optimize. So yeah. The main constraints on while designing are low memory. The biggest problem is low memory, slow processor, deep sleep mode. You certainly can't keep a wake lock on. I did once and I drained 50% of my battery just using my app. Network connectivity change. Like if network connectivity changes, if you are maintaining a Duplex long lift service, long lift connection, it will not work. So you need to design mostly against these constraints. So what I'm going to do is that I'm going to club the first three in a section called designing for 2G. And then I'm going to keep the last point going to take the take up the last point before so as to mention serious problem. So every time your connectivity actually change, you are switching from Wi-Fi to 2G, 2G to 3G, 3G to 4G, anything. Your connection actually closes, but it's not marked as closed or it's not cleanly closed by smack or like whatever library you are using client library. So the thing is you have to reconnect again. You have to close your connection using instant shutdown and then use connect to reconnect again. The biggest problem is every two minutes you have a Wi-Fi switch in India. If you are designing it for 2G, it will go down. It will come up again. And if you are going to reconnect, you are going to get 500 friends again, 100 online offline messages again. So imagine how much is it going to drain your battery? So there are few ways to fix it. Yeah. So here are few optimizations, some tips. Don't fetch all your friend roster is your friend list. So don't fetch all your friend list when you reconnect. Save it locally since you are using mobile. There is a technique called versioning. Implement stream resumption and TLS resumption. That means instead of reestablishing the entire stream, just reuse the previous stream and also you can bypass the SSL handshake. That's very expensive. If you're talking to Gtalk, Gtalk supports a very old version of XMPP. So you need to see if the feature you support, let's say roster or presence, if it's supported by Gtalk or not. So for that, you need to fetch something called entity capabilities. That is what are the capabilities supported by that server? So if you have 500 friends and each have different capabilities, let's say each of them are running different version of client, your client, and you have upgraded something in one particular version. So to support against different versions, you need to get the entity capabilities. It's a good idea to cash it. You can use client state indication if your app is in background. So to tell your server that, hey, I need few packets and you can ignore our club few other packets, then compression and privacy list. It becomes super useful here because XML is very verbose. And when you send streaming XML, your packets are so similar that your size will be much smaller than that. So yeah, the first time you connect to your server, use roster versioning, send version equal to zero, get all your friends, save it locally. The next time you send that request, either you send it manually or it's sent due to change in connectivity, send the version, get only two new friends. It's going to save a lot of your battery. Also, every time you reconnect, you have 14 to 18 steps for minor 2G flick switch. You reconnect again. So make sure that you can use the same stream instead of establishing a new stream. It's true for any TCP, like long-lived TCP service. You can design it that way. And in that case, you just need to make sure that your server stores your stream and thus you can bypass the entire session establishment. The only bad thing about this is that you have incorrect presence. So you are actually online for... So since you're using session assumption, your server stores the stream. So you are actually online for next 24 hours, even though you were offline. So you are basically... Virtually you would never offline. So your friend may think you are offline, but you are actually online. So there's a possibility of incorrect presence. So you need to get it fixed yourself. One more thing, if you are using session assumption, if you are reusing the same session, make sure to reuse TLS resumption. We did the same. We tried using it on 2G, like TLS. It times out very frequently 2KB of certificates transfer every time you reconnect. So, but yeah, so you can use these two mechanisms, but yeah, not implemented anywhere. So it's a good idea to maybe just bypass TLS on XMPP and use some client to client protocol, OTR4j or anything else. Cool. So we already talked about entity capabilities. Gtalk does not... So Gtalk is XMPP and it does not support client state indication, vcard and stream resumption. So vcard, if you want to see the visiting card of your client to, you're talking to client to, maybe Gtalk does not support it and you don't know. If you ask it without checking support, it's going to throw an exception. So it's good idea to check for every client in server what I think they support and for 500 clients is going to probably kill your user's network bandwidth. So it's a good idea to store it locally and add a local receiver to make sure that you clear your data once you are running low on storage, because it's huge. You have 500 friends for every 500 friends, you are storing that much of data. Few more optimizations, true for any TCP connection. If you are running your app in background and if you have many friends, one of them is typing, one of them is sending you an actual message, hey, and few friends are sending you online offline their status. You really don't care if they are typing. So you can just drop that packet. You also, you are okay with getting late presents, like if your friends are online offline or they late or you can get their status late. So you can just bundle them. But real time just so you can tell your server that hey, club divide my messages into three categories, discard one of discard one of them, bundle one of them and the last one send it in real time. So this is something very specific to XMPP, but yeah, you can use it anywhere like for any other protocol. Yeah, again, this packet IQ is huge to 52 bytes, enable Zlib and TLS 202 bytes, send the same packet again 90 bytes. So XMPP one, good thing is XML is verbose, very, very big. But if you use Zlib and TLS properly, your, since it's a stream, your packet size are going to be very small. So just make sure to enable Zlib. Don't enable TLS unless you are sure you will be able to fix resumption. Finally, enable privacy list. As we talk XMPP supports server to server communication. So if a bot is sending you 1000 messages, and if the other server does not support rate control, you will get all those 1000 messages your battery will train your network will train. So that's true for any actual long live connection. Your battery is not dependent or your network usage is not dependent on just you is dependent on whom you are talking to or your peers. So make sure that for any long live service, you block all the services and then selectively white list only few, like in case in this case, we have selectively white listed only server one, client and server three's client. And you need to be sure the server three supports rate control. So if someone is sending me 1000 messages, server three will block on client. You certainly can't block on server to server rate controls. So yeah. So there are a few things we talked about optimizations still not perfect. Still does not work 100% on 2g few things is 357 push notifications. So in Android M you can't keep your service running forever. So you need to use push notification to wake up your app. And then they have something called stream resumption for push notification. So you can use that. But yeah, none of these are implemented. So you have to implement it yourself. And suddenly we need a newer protocol for the system for the design we have discussed here with largest time. By this only we can actually make it work on 2g. Some code pointers. Maxes. I have code pointers at every slides, whatever I have talked about. But so you can have a look at Maxes code. Chat Secure is a very popular app. You can use it to talk to G Talk or you can connect to your G Talk and see how it federates with G Talk server. Then yeah, you can also use conversations to talk to G Talk. So that will be great. And then flow stock. It's the same version of this top except more technical. So yeah, if you want to, if you're looking for implementation, you can look there. That's it. Choose QTT and over XMPP or which one to use when? So actually MQTT is much lighter weight than I mean it's much smaller than like overhead than XMPP. But if you want to build an app where you need more functionalities, like if you want to build an app like Uber, where you need driver's profile or you want PubSub also, you need search also, Jabber search. So XMPP is kind of more complete solution. So if you want, there are 350 or 36 XCPs there. So for almost all your use case, you'll find one XCP. So if you're just trying to build something later to PubSub, I think you should use MQTT. That's much lighter weight. And if you want to do like more search or anything else, then you can use XMPP. Hi. For compression, what other options do we have other than ZLIP? I have used only ZLIP until now. But I am not exactly sure. I need to check but I can tell you after the talk. Thanks.