 OK. Hi. I'm Andrew. I wanted to talk to you today about some experiences we've had at Slack with shipping a cross-platform WebRTC product and some of the things we've learned. So although it's not the main subject of the talk, I thought it would be interesting to go through a brief architecture tour. So you can see we have two clients here connecting to a media server. And all of our calls actually go through a media server, even one-to-one calls, just to step quickly through how a client makes a call in the Slack architecture. It first talks to a region server to understand where it is and which media server it's going to connect to. It then makes some API calls to the Slack REST API where the call state is stored in the same database that the rest of the Slack clients store state in. That API call returns certain IDs in which media server to talk to and tokens and so on. And then the rest of the call set goes through the media server. And so the call is established. The only other Slack-specific interesting thing to note is that some of the client state is updated via the real-time messaging API. But today I mainly want to talk about client-side development and our approach to it. So in terms of client platforms, on Browser, we support Chrome, on desktop, Mac, Windows, and Linux, and on mobile iOS and Android. And so you've seen a diagram similar to this before, but this is just like a super simple breakdown of a WebRTC client stack. You have a UI layer, application logic and signaling layer, and then a native WebRTC layer in the bottom. So as part of this, I'm also assuming you've rolled your own WebRTC binaries. And as we've heard several times, that's not going to be the right path for everyone. But that's the path that we've taken. And that's what I'm going to assume as we go along here. And the part I really want to talk to you about is the middle layer, the application logic and signaling layer, and its API with the UI layer. I love it. So first, a bit of history about Slack audio calls. We launched in April of this year. And one of the platforms we launched on was a Slack Mac native app. And this app is based on Mac app, which is basically a WebKit wrapper. So it's like a single site browser container to run the Slack web app. And the interesting bit here, as far as calling was concerned, is that although it is very browser-like, it didn't have any WebRTC in it. So we already knew having to launch in this platform that we'd be building our own WebRTC C++ binaries. So that was a given. This is a diagram of the split between the browser and the desktop native app implementations. So the top layer is a shared JavaScript UI code layer, which calls down into either JavaScript or C++, depending on whether it's browser or desktop, and further down into WebRTC, either the browser-based one or the native one that we've rolled ourselves. So when we were developing this product, these two were developed in tandem. And that worked fine because it was the same team developing everything. So this pink band I've put on here, which we'll come back a few times, is to represent the application logic. And that's actually, as you can see here, it was spread between the kind of wrapper layer and the UI code layer. And again, that worked fine because it was the same team doing this whole development. We didn't have a very precise interface between the application logic and the UI code, but that was OK. It worked fine. The API between the WebRTC wrapper and the UI code was pretty low level, consisted of a few weekly type methods, passing JSON messages directly from our media server. So again, not really clearly defined layers. Things were kind of spread out. And that worked fine, but trouble appeared as we expanded to mobile because it was not our team any longer developing the UI code. So we had this application logic that was spread out into the UI layer, and the mobile platform teams ended up having to reimplement a lot of that and do a lot of parsing of raw JSON messages from the media server that they might not care about. And so we ended up launching on mobile in this state, but it definitely slowed progress. And adding features later would continue to be costly. So this is a different view based on kind of like an org of the Slack teams. So the calls team is vending the C++ library to the various platform client teams. And again, this is showing like this repeated application logic being reimplemented on by the various teams. So we launched in this state, but afterwards decided to reimagine the approach. So we took a step back and said, what is the ideal cross-platform design in this context? We want ideally to write once and run anywhere. This has a few benefits. You can consolidate your features, and more importantly, probably, your bugs in one place. And it insulates platform developers or client developers from details they shouldn't have to care about. We're working in a very complex domain, and the Java and Objective-C developers probably don't want to have to care about all the WebRTC stuff. So it's great if you can insulate them from that. The second thing is to provide a natural API to the developers, something that they can interact with very easily. So ideally, you would give them something that's exposed in the language they're working with, so platform-specific language bindings. Another thing, another goal we had was to make it strongly typed. So rather than passing JSON messages around, we actually have strong types that we can enforce to let us move a certain class of errors from runtime errors to compile time errors. And so this is just the same diagram, again, moving all the application logic to the wrapper layer that our team is implementing. And again, the view where we do all the stuff that we should be doing. So what did we decide to do? We continued to write the application logic in C++. A few reasons for this. The native WebRTC Libs are in C++, and we're building those, and so whatever we do, we're going to have to be calling down into that language. So keeping it in the same language makes some sense. But also, if you're writing a modern C++, it's actually a fairly painless experience, dare I say, occasionally delightful. If all you remember of C++ is your grandfather's or grandmother's C++, then have another look at it. It's come a long way. So that was the implementation. And on top of that, we had, or we do have, an API written in Dropbox's Genie. This is a tool that Dropbox open sourced in 2014, and it's built for this exact task. It uses a simple interface definition language, which is language agnostic, to generate bindings for Java and Objective-C. And it handles, so it generates Java and Objective-C and C++ on the back end side, and also it handles all the data marshalling between the languages. So it's turned out to work really well. Just before I dive into Genie a bit more, some alternatives we considered. We looked at RPC frameworks. One benefit, or the main benefit, is that they offer the same kind of crass platform API generation and data marshalling. But turned out not really to be a great fit for the domain. It's not designed, they're typically not designed for running in-process, which makes sense, given the usual applications. Maybe GRPC, it might offer this eventually. I think the Java implementation has an in-process server right now, but it's not quite there yet. And another problem is that they can be really heavyweight. Thrift, in particular, generates a lot of classes. And on Android, that can become a problem. And even a Microsoft project called Thrifty intended for Android to reduce the number of classes that get produced by Thrift. OK, so jumping back into Genie, I just wanted to give a quick example of what it looks like. So here's a super simple interface. You can write with two methods, one of them static to actually create the interface. Beside the interface keyword, you can see a plus C. That means this interface will be implemented in C++ and callable from Java and Objective C. And so the second two boxes, you can see the Java generated code and the Objective C generated code. So these are just the interfaces. What you don't see here is a bunch of intermediate code that handles all the data marshalling down to C++ and vice versa. And so for anyone who's had to write JNI by hand before, you can imagine how nice this is to have a tool that just does it. Here's another example. This is a callback interface. So you can see there's a plus O and plus J beside the interface keyword. And this means it'll be implemented in Objective C and Java, and we can call it from C++. And this is the C++ generated code that you get that's callable from your back-end library. So at current count, JNI's generated more than 5,000 lines of boilerplate code for us, which is awesome. And even more important than just the raw number is that as we make changes to the interface, we don't have to sift through all the code and redo all the plumbing and everything. It's just totally done for us. So it's worked out really well. There's a few, I'm going to say caveats, but things we learned along the way. Sorry, there we go. What level of abstraction should you choose for the API? And this might be, it probably is obvious in retrospect, but it wasn't to us at the beginning. But it did become very clear that we should base it around UI events and actions. So there'll be layout differences, of course, in the different platforms. But the essential call UI elements are retained across all the platforms. So it becomes very natural for the platform developers to interact with the library this way. A user makes an action, clicks a button or something, and we call method X on the library. We have some callback method, and that results in the platform code making some UI update. So when you start to think about it in these terms, a natural API emerges from what you're trying to do. This is just an example of some more JNI code. I only put it here to note that we, kind of in a documentation, we have actually UI suggestions along with the events. So the idea being that we encourage, I mean, beyond the design specs, we actually, like when the implementation is happening, we encourage the developers to make the same kinds of UI updates on the various platforms. Another lesson learned is around threading models. Always assume that the UI thread is hitting the library. This might not always be the case, at least, but in our case it is. And so definitely make all methods heavy methods async for this so you're never blocking the UI thread. We ended up deciding to make all methods async just to make the threading model easier. And that's what you see an example of here. So every method call into the library just posts to a background thread. Another threading model type tip is that the JVM gets upset when it's accessed by non-Java created threads. So you need to provide a mechanism for C++ to create Java threads whenever those are going to be calling back into Java land. Some good resources around this stuff. Genie has a project called MX3, which is great for showing best practices. And they even have good implementations of some useful objects, like a thread launcher that handles the creating platform threads that I just discussed and UI thread event loops for posting back to the platform UI threads. Another great resource is the mobile C++ Slack community team. It's a great place to log on and ask some questions. I'm on there. There's some Genie maintainers that are on there, so you can probably get your questions answered if you go there. I wanted to show a quick case study. So this plot is of call survey results for Android clients. So at the end of a random sampling of calls, we give a survey and ask the user what they thought about the call with a few options. I've eliminated everything besides the top two answers. So disconnects are bad, of course, and OK means, like, yeah, the call was fine. We released this library in the 2019 version of the Android client, and you can see that disconnects went down considerably, and at the same time, the OK rating went up quite a bit. And so we weren't expecting this. We were like, hey, what happened? And I attributed it actually to we ended up, part of the application logic that was implemented in the Android UI layer was stuff around reconnections. And this tricky stuff, deciding when to do this properly, we have some very precise rules around it. And so I think when we brought that back into our C++ library and had it implemented by people who were intimately familiar with the details of it, we basically did it better. And this is not to say, hey, those guys are horrible or whatever, but just that when you bring things into your problem domain, you have a better chance of doing it right than having to communicate requirements to people who might not understand them beforehand. So that was a very positive result of launching this. So alternative architectures. As you've heard today, there are a lot of different ways you can approach this, but there are two that are the most exciting in my mind. One is the one I've described today, the C++ and Virginia approach, and the other is hybrid apps using JavaScript and React Native or something like it. And I think the choice comes down to mainly what your existing code base is like. So do you have Java and Objective-C mobile apps already? Then probably what I'm describing now is gonna be a good choice. But if you have a WebRTC web app today and are considering expanding the mobile, then I think the JavaScript and React Native approach is really exciting. Okay, but you might accuse me of being a liar. We'd still have to write it twice because we have this browser implementation and we have to write the JavaScript too. And yeah, that's true. That is true. But we kind of have a plan. It's called Inscripten. And it's a C++ to JavaScript transpiler. We have some patches on top of Genie that will produce and Inscripten bindings for us from the Genie IDL. And we have this kind of hand-wavy plan not actually developed, but if it works, it really will be kind of a right once run anywhere thing. So we haven't developed this yet, but if we end up doing it, I'm sure we'll write a blog post or something, so stay tuned. Thank you.