 Cool. Thank you for having me at meta refresh. It's been so far a really great journey Like thanks to sign up in the whole team like they made it really really straightforward and nice for me to come here And be in Bangalore India for the first time So thanks for having me Yeah, I'm Thomas. I want to talk to you about real-time communication a little specifically on the web platform He I just was introduced already quite nicely, but just to give you a little background on what the hell is this white face on stage I'm a web developer since 98 97 98 I started to do my first website and I'm actually more or less a UX designer That's where I come from and I only learned coding because I wanted to create cool stuff for other people and there was the way to do it I organized JS conf and CSS conf in Singapore and I did it once in Manila So far very popular conferences, maybe you can make it out at some point And I help a lot of startups at jfti and even here in Bangalore. I got The opportunity to talk to a few startups in the last days and there was actually quite a cool experience So the last year I spent a lot of time helping developers Dealing with the technology a web standards that is since 1st of April in large parts finally a web standard a recommendation by the w3c called webRTC and This talk is essentially about everything that I learned about it and everything that my team did To help developers Deal with webRTC and make it usable a lot easier So we're gonna go into the opportunities a little bit what it can enable what it can do how it performs where it works All these kind of things I want to start a little bit with a bit of history into communication Just so you see what webRTC actually enables Real-time communication to be exact now what I'm doing right now to all of you is real-time communication to rate like it's pretty real-time Pretty low latency too. I hope This in a way is real-time communication and as a UX designer This is also the ideal user experience is to talk to somebody that is actually competent and helpful and ideally speaks your language That can solve problems for you. Like this is how most problems were solved 50 hundred years ago, right? You would go somewhere and you would talk to somebody and they would help you So this is what we are all trying to mimic with technology in a way Now in this talk I want to specifically talk about real-time telecommunication Communication meaning there's so much distance between two people that you cannot do it in a person-to-person scenario, right? So what does that mean? Where does it come from and how did it work quite a while back? So this is 1872 and like telecommunication was a service It was like and in large parts of India. I hear that actually still is it's an office that you go to and then you talk To somebody and then they would type it into a device and then it gets sent through a cable from London to the US or Something and they sit somebody on the other side that reads out what happened the data that came from that cable So this is how it all started right telecommunication It's a service Now over the time it progressed a little right it became a thing something that you can actually hold and Interact with and to a lot of people these days. It is still a thing, you know looking at how people use their smartphones these days which in general are Literally general computing devices. They can do anything But most people use him for example Specifically for messaging. It's not that you can't message on a computer Obviously you can but but for a lot of people out whatsapp like that's on my phone my battery dies there Can't communicate anymore So it's still a little bit in our head that communication is a device is something that you walk over in the corner and dial Numbers scratch numbers in and then something that you will hopefully pick up Now Since 2003 I think Skype really made it popular. There was the breakthrough of communications as a software on general computing machines So suddenly Smartphones and laptops and all these kind of things can host communication as an app as a software This is already pretty cool meaning you get into this general computing environment now 2015 changes that again, and you can't see where I'm going to WebRTC is a web standard since 1st of April finally was a lot of effort and time to get to it and WebRTC is a set of JavaScript APIs in the browser to enable peer-to-peer communication, and that's a big deal It's not that there was no embedded communication before you know technologies like flash and Others were able to do this kind of stuff before but you know nobody likes plugins They will proprietary so not as open and there's something about web standards that make them last long Even the web today is still as popular as even more popular obviously as it ever used to be and It is largely because there's a standard meaning it is something that a lot of people agreed on Doing in a certain way and that has a lot of power and a lot of longer live long liberty it Literally goes over century. Sorry tens of years so far, right? HTML was I think invented somewhere end of the 80s so So much about standards right compare this to Symbian Proprietary software it goes away with companies being suddenly unsuccessful now. This is how the web works usually This is how pretty much every web interaction Huge amount of them work you have peers you have consumers you have people in front of their devices and these are the circles And they talk to a server somewhere and then that server relays information to other people right Like I call this the stars stars that up right forms a little star to the server now This is how what you see looks Everybody to everybody you know crazy So you see already a few things right every peer has a higher set of Connections that they need to maintain because it's a connection to everybody that wants to connect to them and there's suddenly no Authority or entity anymore that regulates anything It's just go mad right talk to everybody if you want to send a message to three people You need to send it through three different network connections. So that's peer-to-peer Now what can Peer-to-peer technology be used for right and these are some of the use cases that are specifically enabled by WebRTC Where by TC comes besides its peer-to-peer set up with a set of API's that allow you to access microphone and camera of your devices and with those you can do one-on-one calling Conferencing in game chatting virtual co-working Music streaming like actually we have an internet the office that just build a karaoke app. It's kind of cool Sensor data exchange video consultations even conferences like I attended a conference entirely done through WebRTC This is how web communication used to work in the last years Type in somebody's phone number that you read from a website. It's not really that online Installing Skype and exchanging usernames or inviting somebody on hangout So WebRTC can change all of that These are some of the scenarios that WebRTC can enable and those are just the ones that I came up with or some of our customers actually do Talk to your host you booked an Airbnb or something on travel mob And you want to talk to the guy that you actually rent it from right instead of typing in a phone number do it through the app You buy something on Flipkart when you have a question to the seller in the marketplace offer Like they don't want to share the phone number. It's a privacy thing sometimes And they can control and actually see that you made that call and how long that call was They can't look into the call. They don't have access to that But they can see the duration and other things might help them with conversion rate optimization and other things, right? now in a short WebRTC enables embedded contextual communication like communication audio communication Even video communication or just data exchange where you need it directly to other consumers to other peers Those can be service to actually This is where we add 2015 WebRTC makes communication a feature. It's not a software anymore It's not an app that you go to it's wherever you need it Today in WhatsApp you can call Facebook you can call and these are just communications tools that think it's another form of communication But yeah, you can put it into Ola or Uber and call your driver, right? So this is how WebRTC works conceptually, right? We have WebRTC and I introduced the concept of signaling here meaning there's an authority somehow That does all the introductionary services for you so that two people can actually get to know each other It works a little bit like a LinkedIn introduction or something So you have a known party and both of these peers A and B know this known party They connect to it and because they have a certain token that they both have in common the known party knows Oh, these guys want to talk to each other So I gonna introduce them and then they can exchange information directly You can see already the dashed lines. There's very little information that is actually Transmitted to the known party. It's literally introductory Introduction and the data the heavy weight that goes directly peer-to-peer So five things that make working with WebRTC incredibly difficult today and I go through how we try to solve them With the product suite that we call Skylink On the client side, do you want to create a website that actually leverages WebRTC? To do that you need to do this and That's like extended from what you find on the web that we did this company internally This was like our working draft to figure out how the hell do we get started with this? So it's a bit older already I'll just go through the high level stuff And on the top it introduces the the flow of information that goes between the signaling server the caller and the callee And you have an exchange of candidates which are network addresses and ports to which the different peers can connect to and you have an Kind of a negotiation of the video and audio codex that the different browsers support So it's like a LinkedIn introduction happening through the signaling server You know it through a third party and then the first question you ask is hey, do you speak English? Kind of the same way all of this is done in Skylink.js So you see a lot of you used to pick up the stickers yesterday already, which is very cool It's like oh no sticker. I want that This is what Skylink.js does right Skylink.js is All of that that you saw before on the client side done for you now WebRTC today works in these three browsers But there's subtle difference in it. It's a young standard It's still shifting massively like every browser version that comes out has something new for WebRTC currently and it's pretty hard to keep the track of that and If you maintain a WebRTC implementation yourself, you will constantly need to adapt Well, luckily we do this for you. So we have a project adopted.js that Skylink is also built on top of That shims all the different implementations and different browsers for you So that it becomes equally easy to use in every browser It also actually takes care of some helping functions where we thought hey, this is a bit complicated to use Let's put a function in there. So it becomes easier and it polyfills and it polyfills Because of this WebRTC doesn't work in those browsers yet Or does it? So our company created a little plug-in. It's not the greatest experience. You know, it's not Again, we are back to installing flash or something right, but at least it enables those browsers to support WebRTC And we're working on the experience right now. I just got an email this morning where my colleague wrote me Hey, I found a way where I don't need to force people to restart the browser after installing So it's getting better and better And these work on Mac and Windows So that even IE 9 is suddenly working with WebRTC Now coming to the server side problem number two To you need to create this known party the signaling server, right? So you need at least some kind of instance like a lot of people use Amazon AWS for it Obviously, you need to maintain and pay for it that come costs with it You need to adopt it to exactly the message exchange that you have implemented on the client side so that you know They understand each other And obviously it should be low latency because you want to have this introduction as fast as possible So we have a service for this a backend as a service that does that you can sign up for it. It's currently free That does all of that for you it works obviously seemingly together with Skylink.js so that when you use it You don't the only thing that you need to enter is an API key and it will connect Now so this is something that Pete Hunt the guy that created react set at some point in his talk in Mendeley at my conference and he was like if it doesn't work on mobile That's not worth doing and it's obviously in India at more true than anywhere else It works already an Android on the same browsers and I'll show that in a moment and on top of that We thought we throw in some SDKs for good measure So that you can actually work with it on iOS and native apps and an Android and native apps, too And you can connect those you can connect from an Android app and make a call to somebody on the browser It's no problem And then there are a lot of edge cases webRTC is a very weird technology It depends on so many moving parts to work just right to be able to get a good connection And that is obviously your bandwidth the latency. Where's the signaling service? It closed by how fast is your phone? I mean because you're dealing with audio and video code access encoding and encrypting going on You know it all limits how good your application. Sorry your phone can actually deal with Connections can it do five connections video connections? Can it do like we've gone up to six on a very modern HTC phone That but then the screen gets a bit small actually, you know, can't see six people anymore so well So there are a lot of edge cases and the most common ones are are these two's that a lot of Specifically enterprise customers have firewalls that actually don't let in peer-to-peer connections this kind of hole punching that Your routers at home allow is usually turned off in enterprise systems And then the second one was what I referred to is CPU and bandwidth are just not there just yet specifically on mobile And in those cases you can use services that we provide And make it a bit easier for you like they increase the quality of service of connection establishment between peers because they help identifying a Way to bring those peers together stun does that from the outside looking at your phone and turn does that by being a relay server meaning you actually don't allow It's not peer-to-peer anymore at that moment But you can make a video call and it's going through a server so the latency will be slightly higher depending on where that server is Then location is really important again but it is possible and the SFU is a funky thing think of a conversation that you have with Four other people meaning your computer sends out four videos to four people and receives four videos from four people So it's eight connections. It's eight videos to encode and encrypt and decode decrypt That's a lot of work for a CPU and the SFU essentially reduces this so you send only one video The SFU forward set to everybody that reduces the amount of connections in that specific situations to five So you have three connections that you don't have to deal with meaning the performance goes up Then with requirement goes down meaning you have a better experience Now To the interesting part how does that all work? Does it work in India the big test? I have a lot of links in this That you can check out yourself as well, and I'm going to show you a few of those just so that you You know how it works or not Let's see. So this is Skyling IO This way you can sign up to get your API key and actually use Skyling And this is a tech demo that we did and it's maybe the most common web ITC demo that people share So this is a little website that allows you To just call a friend you send him a link he clicks on the link He's going to a website and suddenly isn't a call with you. So I'll enter a specific address here You can put just whatever behind the slash to create a custom room So I do that now my computer will ask me for camera and microphone access so I should stand here Grant that and Let's see who is there? Hmm now Somebody's in the room Could that be it's me? Oh Feedback so you can see that actually works and it's not too bad You know like the magic here is obviously it's peer-to-peer, right? Meaning I don't go through the internet at all both my devices have a connection to this signaling server But it's only used for messaging meaning the requirements on bandwidth up as low as it can possibly be And somebody already knows how it works feel free to come on stage whoever you are Show your face There you go. So you see like you don't need to install anything There are two billion devices already shipped in on this planet that support web RTC and have a camera and microphone to do this Do we get one more anybody with an Android phone or Mac or Windows computer on Chrome Firefox? Oprah can join the room name is iOS because I'm You know lazy and don't want to type much and I wanted it. This is by the way the iOS app Demo app that we created to show our SDK works There we go, we have a fourth person Nope, we don't but you can see how the peer-to-peer nature of this technology technology is Allowing for great performance because it finds the shortest route to the other peer through the internet and specifically in India Where I hear a lot of people complaining about the fact I Complaining about the fact that The peering here is pretty bad and that service are so expensive in India, and you always rather. Oh No, you killed it By the way, this is working in Safari right which natively doesn't support web RTC So it uses our plug-in to enable all of that. Oh, you really killed it. Fantastic Okay, more examples Let's see if we can kill my chrome too So this is a little cool working app that I created It's serverless meaning it only uses our infrastructure and everything is done peer-to-peer It asked me for access to my camera and microphone and Then asks me hey, you know, thanks, man. Oh your name and I say yo, I'm Thomas Now I'm entering a room and this feels a bit like slack or hip chat or whatever Company internal messenger or chat system you use But I have a picture up here that updates every five seconds Right or if I click the picture so if anybody else goes to that same URL You'll pop up on the side, right? So you can see my entire team sitting there We tried this out with up to 12 people takes a lot of CPU from then on forward because it's sadly not that optimized in the browser Side just yet, but it works And I hear the audio as well meaning you get this kind of feeling you're in a co-working cafe Where there are a lot of people around you and there's a little bit of noise in the background and You can talk to your co-workers and you can share pictures like let's see if there's a picture about meta refresh and thick Flickr by just typing share Metarefresh and I'm sure the picture will be very creepy see there you go meta refresh Just straight from Flickr. So like there's a bit more functionality can mute and unmute yourself and one of the cool things is I don't know if you see this. I maybe I move zoom in here a little the picture actually blurs as soon as I leave the tap So that whenever you're not really looking at this tap Like you get a bit of privacy People can't see if you're in front of your computer, but they can't see what you're doing in the same way if I leave the tab The audio goes down so I'm not distracted by other conversations that are happening in the chat Only if I go back to the tap, I will actually hear full volume again. So this whole thing This whole application was written on top of Skylink and only Skylink there's no jQuery no react no backbone No, nothing it uses Skylink and I use a helper library here to do the blurring and grayscale conversion of pictures that I scraped from Stack Overflow essentially and This whole thing is 420 lines of code with HTML. I put the JavaScript even in the same file I didn't care about making this an include or anything. It's really tiny doesn't have a service site It saves my chat history in local storage. So whatever you witnessed is replayed when you enter the room again It's really really straightforward. So you can write these kind of serverless Lightweight applications that use distributed storage to maintain state It's not as straightforward as it sounds, but you can Another example that I hope works. It depends always on how good. Oh, hi, Roshnikant Isn't that an actor? Just checking So let's see Now that looks good This is another stupid thing that I wrote and This is a selfie remote Like I come from Asia like the real Asia and people there use Selfie sticks to take like fancy pictures of them in front of ridiculous backgrounds and Once you put your phone on the selfie stick like we have one problem Like a big one, which is you can't reach the photo button anymore And now they have Bluetooth solutions and like even like mechanical fingers that tap and all this kind of stuff I thought I'd create a web RTC solution for that. So I visit a website on my phone that is already here and I need to have the same public IP address from My laptop which I hope is the case Otherwise, I won't be introduced because I use the public IP address of this internet connection as the room identifier So that our signaling server knows. Okay, two people with the same IP address. I'm going to introduce those two to each other And I just hope that works Here it says waiting for remote. He had says waiting for phone. So that tells me like do you load balance over multiple connections? Okay, and then you have session stickiness Okay, then that's not gonna work Meaning my Android phone has different public IP address than my laptop. So I I would need to hack it now Right, but my other device iOS. So there the website doesn't work yet All right Yeah, sure if you want you can go on your Android phone to this address But you need to add phone dot HTML behind Maybe I can actually show it to you just in two tabs instead of on two devices You get the gist So here I visit the phone Please share access to your camera If you let me come on Google. So here's my picture, right? And this happens runs on the phone I'll put this here on the right side. Why not? and This happens here Here on the left side. So I see a video stream of myself being transferred to my other device, right that I would hold secretly under the picture and I can capture a picture and so I use webRTC to Load the preview over so this is a video stream And that picture is actually not taken out of the video stream on this side But it is captured in full resolution on a JavaScript canvas on the phone and then saved as a JPEG and Send over via the data channel of webRTC to the other side So I have a high resolution picture here, although the preview can be pixelated and shitty connection It's also an advantage because the video streams are optimized for low latency Meaning if the codecs figure out there's shitty connection They reduce the frame rate and sometimes the resolution too. So the video favors Low latency and real-time over quality of the video, right? So it's more important that you see things moving and you hear each other no matter how crappy Because you can't just buffer. It's like really bad in a conversation where you get stuck And then eventually continue not that helpful, but the data channel transfer can go very very slowly So it can take like a minute to transfer the full resolution picture from the phone back to your other device, right? But you don't care. You just want the full resolution picture. You want the quality. You don't want it as fast as possible so This is like I think using the right tool for the right problem Taking selfies so we got that covered with webRTC very important Here's an interesting one. So for those people that work with React We have a react sync react is awesome. It brings state back to the web in the 90s. There was always this One URL is one website Right and whenever you open the same URL you get the same website eventually all this like Pearl PHP and Service site dynamic languages came up and it was not true anymore because personalization and all this kind of stuff Now with react there suddenly state again on the client side meaning You have the same state in your app. It will render the same way now. We can actually use that really well to connect to Pierce via webRTC and I have a little demo running here on my local host With this Just to show you how that works. I've used the same website on both sides and this website actually before I do this This website just renders whatever I type into the text box on the left It renders and mark down on the right so I can like You know do Links and all that kind of shit Sorry rather straightforward. It's a very very small react component I Opened this in a new tab Here's what will happen They connect via webRTC. It's two lines of code that I added to this react component It's a react mix in that I wrote called react webRTC sync You add an API key from Skylink and a room identifier and now one way one side will send the state over to the other side So when I keep writing here, and I make maybe a headline Come on. Oh wait, I haven't used this in a while. I better start over and depending on the quality of the internet connection Ah, here we go. So you can see how on the other Side and renders the exact same way and this You know, that's not going through a server and like doing another set of persistency Like all it sends over is the visual state react only as a DOM rendering engine So it's not really a JavaScript framework that does all this server communication and so on So it's just for DOM rendering meaning it's really just what you see It's like screen sharing but instead of sending pixel data. It sends a JSON object Right, so it can be much much faster and because it's peer-to-peer. It's super low latency Like it can't get faster literally and You can do product demos with it Right if you have if your app is entirely written in react try out in every component You add these two lines of code to enable it for sync and then you visit the same website on another browser and Whatever you do in one side will happen on the other side That's pretty cool. I thought Like we have a lot more of these demos I have a demo where I build a a web component like we had this talk earlier that essentially enables you to do a call So it's one tag. I call it skyling call tag You put your app key in here call ID is who you are Call our name is your name call II is who you want to call and call the name Is the name of the person you want to call you can put as many buttons of this as you want on your website and Style them however you want and I show you the little demo here as well It's massive websites. You'll see this. It's crazy complexity and the design See the internet doesn't let me down. They reconnect really quick now. It doesn't want me right now So all this website gives you is like a button one says call Alice and the other one call says call bar If I click them what will happen is that there will be a video rendered in the lower right corner and As soon as the other side picks up they're going to be a pop-up message saying hey Alice is calling Do you want to take that call and you say okay? It's gonna establish a video call and it's one HTML tag that you add Done with polymer. We heard the talk before Would love to show it, but I don't seem to be too lucky Well, so far everything went well There you go So I think this all is skylink you find more on like all these demos are linked on this page From get a room over the co-working chat the selfie demo react sync their tutorials on how to build a simple messaging service That just does chat and all this kind of stuff It's pretty straightforward. It works with your phones We're gonna have the karaoke app in the store soon Where it uses your local Wi-Fi to turn every phone into a microphone so that everybody has a microphone and can sing And if the website that is the master control you get like for every phone that's connected a little volume slide So you can mute people that sing really bad You know, it's a funky little app. There's no Real purpose behind it other than being fun and like trying out what you can do with low latency web ITC And it uses the same kind of thing where the public IP address is your room identifier So only people in the same Wi-Fi will be connected So you don't need to worry about some creep from somewhere John suddenly singing with you That is what it is. So that's Skyling. This is like the real-time communications working on mobile and an Android and I hope I can Give you one last inspiration on where web ITC can be going now Like I said the problems in India with connectivity and service are outside much cheaper and so on you want low latency Wherever possible like speed is the biggest engagement driver on the internet and that's not entirely true speed It's the second biggest engagement driver on the internet. It's only surpassed by perceived speed, but that's a whole different talk To achieve that you can use web ITC in every classroom scenario teacher wants to show somebody on the students app Why go to a server somewhere in California or at least Singapore and come back through the same internet connection to the students To update whatever it is you're doing specifically the data gets media heavy Really use rubber to see for that it creates the same connection between two devices right away on the same Wi-Fi you get Two milliseconds maximum latency because that's how good Wi-Fi is You can't be that right and even on a country scale like if you know your service are somewhere really remotely located you can load Big content data you can load states if they're Distributedly stored or even search results like there's a guy in California who implements bit torrent currently as a website So you could essentially go to a website and will stream videos from other people just like you visiting the same website no app installed No need for it. So how cool is that and I think with that the web can be a very different place ten years from now where we rely less of on service and much more on each other I Carefully call it democratization of the connectivity if you will and I hope that catches on it's also Will surpass a lot of the NSA and surveillance problems that we have because every peer-to-peer connection is the shortest route possible That's how the internet works meaning it will have left Sorry, it will have less surface area to servers anywhere So there's less opportunity to take that data out and then it's still encrypted by default So that makes conversations happening peer-to-peer very very private. I Think that's something to strive for so thank you for your time questions obviously Hi Thomas. Hi. So I used to work in a wipe and cloud communication company Yeah, so been working with web RTC for the last one and a half two years Yeah, so we implemented a web RTC click to call widget a Chrome extension Let's you make calls between each other But and because it was why be integrated with our existing white gateways. Yeah So our main problem happened when you have a client with multiple virtual networks network adapters Yeah, the ice negotiation would take a long long time. It would take up to five or six seconds Yeah, I mean I'm sure you must have faced the same problem. What's your take on that? How did you know so we're ready to see these days supports trickle ice which essentially sends whatever port and IP combination it can find that works as fast as it can find it that's in 90% of the cases solve that problem But it cannot always solve that problem like that's one of the problems They are timeouts involved to for network interface lookups and all this stuff That you can only surround that good. So if a firewall or a network is badly configured, there's not much you can do The best thing you can do is use trickle ice Right doing that. But when I read up last they were talking about just using the most the active adapter for the one Which was used last and then rely on that because if you're using something like VM where and stuff like that Sure, just goes deep inside and it's in a hole. It's not coming. That's correct You can obviously once after you have successfully established one connection you can cash the candidate and send that with a priority Any plans on growing this and connecting to the existing phone system? So let's say doing this across many plans. Yes, I can't say much about the progress of it just yet It's going well, but that's all I can say It will come yeah, but I hear it also somebody told me that calling from a General computing machine to phone numbers actually legal in India No, it's not we were doing it. It's still doing it. Okay, so you see the demo finally working, right? We saw the call happening and now Alice and Bob can finally talk to each other That's the web component More questions Okay, you're all blown away. Well go to Skyling sign up get an API key. That's great. I love it Catch me later outside. I actually thought I'll take a lightning talk later as well We will go a little bit about this whole do I do an app do I do a mobile website? Thing like just to throw in more gasoline on the fire