 Hi, I'm Zedr DuMain. I work at Interactive Intelligence in Raleigh, North Carolina. We have an Ember application that we've been working on for a few years. It started pre-10 and is now Ember 2.4 already. So we're pretty excited about that. And for the past year, I've been working on building WebRTC into our Ember application to provide it with voice and video capabilities. And during that last year, learning both Ember and WebRTC, I learned a lot of the different things that we need to kind of take into account in order to make an application that's maintainable by humans. Because what I learned pretty early on is that you can really quickly create a mess. And one of the things Ember is really great at is helping you avoid that mess if you have a pattern in mind and are able to kind of set out a plan. So what is WebRTC? WebRTC is, at the most basic level, a collection of standardized APIs for establishing voice, video, and data connections between peers. So what does that really mean? It's basically a set of browser APIs or native APIs, whichever your platform may be, that provide you the ability to build apps from a high level on top of an entire stack of underlying protocols. So using ICE to collect peers to establish connections across networks. Creating UDP streams, TCP streams in order to share binary data or video data, media data. And then having end-to-end encryption for that, is basically the entire breadth that WebRTC gives you. And there's lots of different applications for this. I'll go into that a little bit more. So this is an overview of what a basic WebRTC connection looks like. And this is really complex, especially when you take into account the fact that normally your browser APIs don't really have a lot of statefulness or complexity to them. So I like to say that WebRTC is changing the native API paradigm in the browser. You get highly complex, discrete, and continuous event-based stateful objects. What that really means is normally you'd use your browser APIs to schedule a function to run, or call some function that returns a promise, or basically just access something in the DOM. Even with something complex like audio or video, you're really just dealing with a single object that might have a state and some properties. When you have an HTML5 video element, you can play and pause that element. You can change the source. You can change volume properties, things like that. But with a peer connection, it's very stateful and it's highly complex. So I'm not going to go over this entire diagram. I just wanted to use it to illustrate the complexity of that stateful connection. So the three main APIs that we're looking at there are GetUserMedia, which allows you to actually access hardware media from the browser. Things like your webcam and microphone. RTZPeerConnection, which is basically the big grand box of establishing that connection between peers. And then RTCDataChannel, which allows you to exchange arbitrary binary or JSON data across the connection between peers. And to boil it down into these three pieces seems overly simplistic. It really is, there's a lot of complexity to this. And in order to get into some of the more advanced techniques and topics, ORTC and then the newer 1.0 version of WebRTC have been released. Which kind of address a lot of that. But I'm not going to go into that very much today, because I want to talk about dealing with WebRTC with Ember. So this diagram, this stateful diagram is what you see all just from that one RTZPeerConnection object in the browser. So quirks and obstacles for working with WebRTC and Ember. One of the main ones I encountered early on is RerenderDom. And I'll show an example of how that can be a gotcha here in a minute. Ember apps are long lived. When you're creating peer connections, exchanging media between browsers, you really run into this problem of, your browser isn't necessarily the best at cleaning things up, especially highly memory and CPU intensive objects. So how do you get around that and how do you deal with that? You're also dealing with discrete event-based APIs. Ember is really great at doing bindings to things. But what happens when you've got lots of events flowing, you don't have a single object you can bind a state to? You also then have complex state and state transitions. It's not just enough to bind to the discrete state of something. You also have to know, what is that state transitioning from? What's it transitioning to? You also have to deal with browser permissions. Dealing with, can your app run on HTTPS? Can it run on HTTP? What if it's running and the user declines permission to the video camera, but grants permission to the microphone? So you're dealing with permissions and your app doesn't always function how you would like it to. And then obviously dealing with humans. Humans are really great at detecting errors with media. Anytime you've ever watched a YouTube video or a Netflix video and it buffers even for a split second, humans are great at noticing that. And that's really terrible for building apps because it makes it hard. So dealing with all of these things are the basic challenges. And then the other aspect of the human challenge is your developers. If you're not thinking about the developer experience, then you're not really thinking about code. So you really need to know how your developers are going to be able to come into a project that's got WebRTC and Ember and really be a functioning part of that team and be able to understand that complex code base. So we'll start off with just a really basic demo here of the HTML5 video quirk. So what I'm going to show here is there's two boxes, a blue box and a red box. And when I click start, I'm going to get access to my webcam. And when I get access to my webcam, I'm going to attach that webcam stream to the DOM in both the blue and the red boxes. So I click start and there I am, great. So the re-rendered component issue is one that I ran into really early on. And as a really basic example, imagine something like Google Hangouts, where you have different video participants displayed on the screen. And something happens, say one of them begins speaking, you want to take their video and promote it to be the primary video. Well, what happens if you do that in Ember and you re-render the component? So I'm going to simulate an Ember re-render here, is that, uh-oh, one of those streams gets frozen. So why did that happen? This is because in HTML5, in every browser that supports video, we'll actually pause the video element if you move it in the DOM. So that's really unfortunate when Ember is really great at re-rendering things all the time. So you can, there are little ways to hack around this, but it really depends on what your humans notice. If the people using your application are really adept at noticing these pauses as mine are, then you have to come up with some alternate solution to prevent the re-render instead of working around it. One of the things we tried really early on was just having programmatically, whenever the Ember component would re-render, we would just programmatically play the HTML5 video element. But that delay still was too much. Our users noticed it, it caused a blip in the audio. And it was really just too many problems there. So I will kill this. And I really don't have a great solution to this. At this point, I like it when I go to a talk and somebody says, here's a problem, here's how I solved it. But I think I like it a little bit more when I go to a talk and somebody says, here's a problem, please help me solve it. Well, this is me asking you for help solving this problem. Right now, I'm basically just using jQuery to ensure that my elements get played if I want to allow the re-render. And if I don't allow the re-render, then I'm using CSS tricks to move things around. As opposed to actually using Ember for all of its strengths. Neither of which are optimal solutions. So help me solve this problem. So there's a bunch of layers of concern with your Ember application with WebRTC, and I want to kind of go through my opinionated approach to structuring your application in a way that helps you maintain this over time and maintain it in a way that other developers can jump into your code base, understand what's going on, and contribute back to it. The three main layers of concern that I have separated are the session manager, the service, and your components. Session manager is not something that is baked into Ember. This is something that has kind of emerged or is emerging as a pattern in WebRTC libraries. This is something that normalizes browser API as native events across browsers. Right now, WebRTC is still pre-1.0. That means that multiple browsers support it, none of them are the same. So you don't want to have to, in your Ember application, code for those differences, especially since it's already been done. You also don't want to have to deal with hooking up all of those native events and adding your event listeners. So there's lots of different session manager libraries out there. One that I have used in the past is Simple WebRTC. I contribute to that one on GitHub. I've also used Jingle.js, as it is a signaling layer. And with some of the extra libraries out there can provide your sessions. Basically, whatever you want to use is great. Provided you don't try to all code this from hand in your Ember app. This is not something that I would recommend because it gets highly complex and it muddles up your code base and it doesn't allow you to separate your concerns. The next is your service. Your services, your kind of vanilla Ember service, this allows you to create your session manager and manage it. It also allows you to manage your signaling channel. And I'll talk about that in a second. And then you also want to tie your session state and say it transitions into your application. And I'll show a way that I use my service to do that in kind of my opinionated approach, acting as an event bus and gateway. And then lastly, your components. Your components are important for rendering and interacting with the application. And it's also useful for managing UI state. And I have an exclamation point there because it's usually a red flag if you're dealing with UI state. But I'll show how it's kind of relevant here. So let me back up one. So for the service, this item managing the signaling channel connection. When you have a WebRTC application, you have to exchange information between peers in order to establish the connection. And most commonly, this is going to be a WebSocket. So if you already have an active WebSocket in your Ember app, hopefully it's already managed by a service. If you don't have one, this would be the place that you would create that. You definitely don't want to be creating WebSocket connections in your component layer. And if you do it in your session manager, which some session manager libraries out there have the ability to do, you kind of lose a lot of control that is beneficial to you. So your service is, this managing signaling layer is really just opening a WebSocket connection to your server in order to exchange your signaling information to establish your connection. So Ember evented service as an event bus. I'm a big fan of Ember evented, and I don't see it used as often as I would like. I think it's really powerful. And I kind of want to show how we can use it to act as an event bus. So let's say we have a component, and we want to handle some event. Say this is a message over our WebSocket. We've received a message, and we want to be able to process that at our component layer. Well, if our service is managing that, what we can do is, our service can extend Ember evented. And then when that event happens, we can just trigger some event and pass the event data. And then in our component, when our component is initialized, it grabs the service, which would be injected. And then on some event, binds a handler to it. So then whenever this event happens at the service layer, you can surface that up to your component layer. And what that really allows you to do is have components that are short lived, but are reacting to events at the service layer. In order to ensure that your events get to the right place, but aren't managed at a place that is short lived. Adding a more complex example to this, what I have here is showing that, let's say there's a foo event on the service. What I can do in order to create, change this event bus, or really just bubbling up events from my service to my components, is I can also use it as what I kind of call it like a gateway. So in my service, I can look at my store and find a person record associated with that event. And then when I find that, then I can trigger the event and pass that person object, that person record, as an argument to that event handler. Then the component doesn't have to do any lookups to find the person record associated with that event. It automatically can handle the event at the same time as it has access to that person. This way, it really keeps all of that state, with be this Ember data or whatever, at the same layer as your service connection. All right, so from here, I'm going to switch over to an example application that I put together and kind of show how we can build something real with this. So this is an example app, Ember app running, just a basic demo kind of loosely based on what our service platform is. But what I have here is, for example, a website where I want to provide the customer the ability to talk to an agent. So they land on this website, you know, this is an Ember app. We've got a couple of different routes. You can see as the route goes through, we kind of maintain some of the core application render, but they're still able to switch around. So looking at the code for this, we have a service for video chat that extends Ember event as in my example. And in the init function for the service, I create my session manager object. In this case, I'm using simple webRTC. So that was that point where I said our service is responsible for maintaining the session manager. It wires up a few events for the session manager to some handlers, and then it provides some basic functionality. It allows us to join a room, to leave a room, connect to a room, basic operations that you would do for your kind of chat room session. And then in our component, you can see when the component is initialized, we wire up a few event handlers for our video service. So we actually grab that service that gets injected using Ember Inject, and it wires up local media started, video added, video removed. These basic events that are going to happen really discreetly on your signaling channel with your webRTC connection to your component so that you can actually render them at that level. So, and it has a couple actions, join video, leave video. So if we look in the app, what we have is the application rendered. This is the route render here. You can see as we switch through, the route is re-rendered in the middle. And then at the high level, we have the video chat component rendered at the application level. And that's important for a couple reasons. The primary one being that what you don't want to happen is when your connection is open, if your user is gonna switch around between the app and switch to different routes. You don't wanna destroy that component. It's really important that that stays rendered the entire time. Otherwise, your connection is gonna fail and your user's gonna be booted out of their video chat session. So it's rendered at a high level that we know isn't gonna be destroyed when we switch around between routes. So when I click the button, a couple things happen. It first makes a request to grab user media. It then levels that up to the component, and the component is able to render that to the DOM. So looking at what happens there, we have a local media started event, which gets started from our service. From there, we schedule after render. This is that example I was talking about earlier, where we're doing some things with jQuery. It's not ideal. We'd love to be able to do this in number. After render, we grab that video element and we attach the stream to it in order to render it in the DOM. So why was that event done? Why did we have to look at the service to grab that event? Why couldn't we just do that in the component itself? Well, the reason that's important is that we're allowing our session manager to provide the APIs for doing that. If we're doing that in the component level, we have to manage all of the differences between browsers. We have to handle permissions, all of that complexity that we really wanna keep leveled at our separate layer of concern. So in our component, when we ask for media, so we say join video, when we join the room, we go to our service and that provides us the ability to start local media and then load it up. So the next thing that happens, if we open this in a separate tab, is say another user joins and they want to talk, be a part of that conversation as well. So they do the same thing. All right, so what happens when we want the second person to join? The second person comes in and does basically the same thing in the UI. Now granted in real life, they would probably be in a different type of UI if this is a customer to an agent. But for the sake of simplicity, they're gonna be in the same UI. So the other customer comes in here and does the same thing. And what we see are that now the second video stream is established. And we can actually see that a real WebRTC connection is happening here. If we open Chrome's WebRTC internals, we can look and see the actual real life WebRTC connections that are established. You can see I've got two, cuz I have two tabs open and one's connected to the other. So what happens there? In our session manager, in the service, because we have bound to the created peer events, the peer stream add-in, peer stream removed, what we can see is that in our service, we can say, look at our video added event that we've bound to and let's just trigger that, to keep it simple here, we're just triggering the peer directly. But what we could do, you can see is, like we said in the example earlier, was look at the store, find a person object associated with that peer, and then emit the event with that person record. So that the component can, maybe it displays a name associated with the peer connection, maybe it displays a profile picture, whatever it may be. And then the UI can render that more friendly way. For the sake of simplicity, we're just gonna emit the peer event. So when that peer event is triggered on the component, the video added element or the video added event is bound to that video added event on the service. So the video added event occurs, we've got the person record. We're gonna add an object for that person record, and then after render, we attach their stream to the DOM as well. So that's your basic example of using your session manager to emit events that your service can act as a gateway and a service bus, to leverage or to surface those events up to the component. From there, you can really get much more complex. You can start binding to data channel events. You can bind to peer shame removed. So we can see if this tab decides to hang up, the other tab can bind to that. And it would be able to remove that event if we bound to the video removed. All kinds of different events occur. And as you get deeper into the complexity of WebRTC, there's a lot more that you can do. And it's important that you have these separate layers of concern. Because if you don't, you're gonna have a massive component that's doing browser shims, you're gonna have event handlers, you're gonna have media, you're gonna have all of these different things in your component. And it's just a big problem for code maintenance and human readability. So that's the basics of it. And what we want to be sure is that you're acknowledging that there's all these different gotchas. And then there's all these different things that can kind of go wrong. And in the past year, I've seen all of these things and more go wrong. But with the help of Ember being able to have a sane way to separate my layers of concerns using services and components. And then having the ability to bind my UI state. And then having the ability to handle my permissions in a way that I can surface the UI changes only once the events are ready. Really makes it useful for building a WebRTC application. So thank you and I appreciate the opportunity.