 We've got Fipo someone might even know him as Philip Anke Okay, did I say it right? Yes, you did he comes to us from appearing and Others might know him by the guy that has the most number of bugs in the chromium whatever about web RTC I know him as the guy that if you have any real issue of the behavior of web RTC He's the guy to ask you ask you get immediately your L of a bug in Chrome That is the responsibility and in which versions it was Published what happens today and when it's going to be fixed and today is going to talk about Duo one of the things that he loves doing is to tinker with stuff and it incurs usually with others applications that use web RTC So yeah, so today I'm going to talk about duo thanks side for the intro so I can skip that So last year We did a web RTC hacks series chat asked me after reverse engineering hangouts to do some more stuff on web RTC hacks and we got Google to sponsor a basically a survey of the mobile application space we did stuff on whatsapp on FaceTime Facebook Messenger and Why did Google do that? We didn't find out that last year, but at Google I owe we saw them launching duo and It's built by the team behind web RTC. They said and they said we know this technology like no one else and Yep challenge accepted So I started reverse engineering that but first why would you do that? So one of the things is you want to look at what your competitor does? I'm not I'm playing I'm competing with Google here and you want also to look at how your own app behaves and Like what protocols do they use for signaling? What codex do they use do they use h.264 vp8 vp9 what bandwidth settings do they use which you will see in my period a lot We do tinker with that What signaling protocols I used what how does the connection establishment work? We saw that was what's up and do they have any exciting features that you can? Let's say borrow So in terms of tools I prefer just to sniff the traffic on the network It's easy on the legal side. You don't have to reverse engineers binary, which is a little more tricky usually and I mainly use TCP dump and wire shark tcp dump for capturing stuff and then wire shark for analyzing it Even though I end up writing a lot of Actual codes that processes the packets and then displays them in certain ways. We'll see that So I also likes is Wi-Fi pineapple router it's very convenient because it's a Wi-Fi hacking device and It's very convenient for mobile development because you can just SSH into it do a TCP dump and get The same behavior you get on wire sharks the same functionality on this device and it allows you to capture from your mobile device So duo has some Interesting features first thing is the connection setup Google was very interested in that last year The other is knock knock which basically shows the callers video before you accept and Another big thing they are proud of is the switch between 4g and Wi-Fi So in terms of connection setup we found when looking at what's up that they are going for a relay server first and Basically they start doing that call establishing a connection to the relay server and then once they get that That is most likely going to work. They try to move to peer-to-peer and That gives you a slightly faster call setup since you have less connections to check and if you use ice there's a slight cost of doing more connections checking more connections and Yep, we found that duo uses turn first. So the orange line you see here is On port 19 three or five they're running the turn servers on the same port as they do for their upper eC demo and What you can see is also that it seems to be used exclusively during knock knock which might be a privacy thing like you don't want to expose the other IP address and once The knock knock faces over which took like seven seconds in that graph you can see the pink line and You can see that the other side is answering It is the audio you can see at the bottom. It's a bit late. I don't know why yet, but maybe I'll find out and It uses detail s it uses eCDS a p 265 to Generate those certificates which is much faster than the old RSA Certificates on mobile devices and that was an optimization that started in 2014 you can see that the If you look at the wire shark stuff, you can see that it's 91 bytes Certificates hidden inside turn channel data, which makes it very hard to find and initially I didn't see it I thought oh, it's sts, but it wasn't So knock knock it is not a really new idea. You see who's calling and then you can decide whether you want to accept And it has probably been tried many times in the past but in isn't common So why didn't it work out? I think it requires a very high video quality Instantly in the first few seconds because if you see a blob of pixels, then you're not going to accept the call So what Google did is they implemented a new way to estimate the bandwidth And that's called sandside Bwe and you can see it in the RTP packets because they have a specific RTP headache Dension with an ID 5 and I could observe them So the orange line there shows the ramp up to 1 megabit per second And I think Nicholas is going to talk about that later and there it takes 7 seconds, which is a bit bad probably I'm going to touch that later, but you see the answer is video the purple line goes up very fast and less than a second and I Blame those results on the capture setup. I had So one thing I noticed is that after knock knock one part of the ice username fragment changes that is similar to an ice restart, but it's not a full ice restart and We'll talk more about that now because the seamless handoff they have it is it's quite a minute Rest this feature basically you think situation is you walk out of the door you walk out of the range of your Wi-Fi and then just switches from Wi-Fi network to mobile network and It works really great. I found it to switch in less than two seconds. So in terms of usability, it's great. I Even saw it sometimes happening between two clients on the same Wi-Fi Which then fall back to the mobile network, which probably is a bug So how do you capture that? How do you capture stuff from the mobile network to observe this behavior? So I work for a telco. So appearing is part of tenor So I just thought okay. Maybe I can do my pineapple stuff on the one end and a GSM gateway support note box on the other end It's a little hard to get access to those boxes unless you're a telco even then it's not easy and It turns out that it is very hard to sync those captures between those two capture files you get from that Fortunately, there's an easier way. Basically you capture the receiving end of the call So the other side is initially you connected to 4g and Wi-Fi and then at some point you just plug C unplug the Wi-Fi router and What do we see when we do that? we see that on the left side there is the Orange line that is the turn first part before knock knock is accepted Then we can see that the pink line where peer-to-peer kicks in it's on a different connection And then at 1615 I unplug the router. You see the pink line ends and There's this orange part in the middle which kicks in after that It's on the same connection that was used previously with the turn server But it's a different peer you can see that if you look very closely in the captured packets It's not very nice because you need to go through a thousand of packets and then after plugging the router back the traffic switches back to the Wi-Fi network because in some countries it's costing you money to use the mobile and data and We can also see During this switching phase when the there's a slight glitch when turning the Wi-Fi off that forward error correction is kicking in So you can see that in those red and green lines at that point and that turned out to be important later and It's the same connection as we've used previously It's turned you to the capture setup I had and you can see the different peers in the turn channel binding and send data indications and What you we can see here is the orange line Continues even though the pink line is Transferring all the data and most important. It's not an ice restart the username fragment combinations stays the same Which makes it by definition not an ice restart So ice restarts are basically the standard way of dealing with a connection failure in ice You send gather new candidates and send them via to the signaling channel problem is if you walk out of the door your signaling channel may have just died as well and What do it does it avoids this it doesn't do an ice restart it keeps this Other connection open and then it can seamlessly switch over to that connection That has the advantage that it works when the signaling channel is down even though the signaling channel is probably quick so It might reestablish at that point It's very hard to determine and I don't think they're going to talk about that anytime soon So another thing to commonly check here is what audio codec is used like we know Google still likes the icic codec a lot It's still in chrome But in the packets in the dump I just found payload type 111 which is the standard opus Payload type no surprise here. I mean opus has a great quality The more interesting question and I'm looking at utai is what is the video codec used is it h.264? Is it VP8? Is it VP9? So I searched for these standard payload types which are 104 VP8 101 for VP9 and 107 for h.264 and I didn't find anything and that is why it defaults the WPC library encodes the data or sends them prefers to send them on the network using the redundant data, which is payload type 116 However, there were retransmissions during that glitch phase and it took me quite a while to find those and What I saw was a number of packets with a RTP payload type 99 which in chrome is a common retransmission payload type for the payload type 107 which if you paid attention on the previous slide turns out to be h.264 and I Think it's because I captured iOS iOS sessions mainly and of course it's on iOS It's a battery life question and you want to conserve battery life. You want to use hardware acceleration So h.264 is a natural choice there it seems that the Controlling the bit rate of the hardware encoder is a bit tricky We've seen a number of bug fixes there recently and the quality I got from do was less than I expected so more improvements Overall, I think do a soft quite a number of hard engineering problems and I've seen the problems I've known in the last two three years that they have been Working on it and it really took them that long probably So to do this fast call setup They do this turn first stuff which landed in chrome in January, I think they do easy a dsa certificates weaponry see switch Completely to that. It's the default in chrome since chrome 52 This walk out of the door problem is solved by keeping a fallback connection open and The hardware encoding to maximize battery life Using h.264 for iOS iOS that is something that has been worked on last year mostly and One of the big problems that still remains is this initial video quality seeing and you really want this fast random pop due to Sandside bandwidth estimation, and I think it is very promising What is going to happen here, and it's going to be default in chrome 55, which is Going to be rolling out in a couple of weeks So basically that makes do a real benchmark against which you can test your application in terms of quality and features And I'll let just let that stand so good work Justin. Hope you're not unhappy