 All right. Hello. Hello everyone. Today, Leonard and I will be speaking about how we can get a bit more diversity in the WebRTC ecosystem and how alternative WebRTC implementations will help us to achieve this goal. So I'm Jeremy. I'm the CTO of Spassinov. I've been involved in free software since 2000 and actively using Python since 2007. And today my perspective on this is as the author of AIO RTC, Python implementation of WebRTC. So hi everyone from me as well. I'm Leonard Graal. I do enjoy network programming. I am the author of raw RTC, which is a partial WebRTC implementation of salty RTC, which is an enter encrypted signaling solution. I am apparently a W3C WebRTC invited expert. And I do work for Threema when I don't work on my personal projects. So WebRTC in short is about secure peer-to-peer communications, whether you're exchanging audio, video or data. And if you've made use of WebRTC, you most likely have made use of the WebRTC.org codebase, which is a project which is driven by Google and which serves as the de facto reference implementation for WebRTC. It's a codebase which is widely used by browsers. Chrome uses it extensively. Firefox also uses it at least for the media part, less so for data channels. And with Edge converging towards Chromium, well, you know it, Edge will be using the same stack too. WebRTC.org codebase deserves a lot of credit because it has put WebRTC into the hands of millions of people. So we probably wouldn't be talking about WebRTC today if it wasn't the WebRTC.org codebase. Nevertheless, if you've tried to integrate this codebase into your own custom project, so we're probably speaking outside the browser space, you will probably have noticed that this is a massive project and integrating it is hard. Tracking releases is also quite a challenge. And to give you a ballpark figure about what I mean by a large project, here is a recent tweet by someone who should know. So Justin said it's weighing in at 1.2 million lines of code. So I don't know how many of you have written that kind of code, but I have not. And so what can we do starting from here? Do you have any other choices if you want a library which supports WebRTC for your own project? The answer is luckily yes. So we're going to talk about two different libraries. For my part, I'm going to talk about AIO RTC. It is a WebRTC implementation which is written in Python. It leverages the modern Python support for Async.io. It supports audio, video, and data channels, so it's got you pretty well covered. And weighing in at around 6,000 lines of code, it is much more pleasurable to hack on. And we're lucky enough to have reached full test coverage on it. AIO RTC started its life as a testing tool to test the availability of my company's WebRTC endpoint, and it has grown significantly since then. As a Python project, one of the key selling points is that you can tap into the broad Python ecosystem. For its audio and video frames, AIO RTC relies on the Pi AV project's audio and video frames. This is a binding to FFMPEG, so this gives you a lot of power, whether it's in terms of reading media from various sources, whether these are MP4 files or an RTSP stream. It's got you covered, and it also gives you a lot of possibilities in how you output these media streams. The Python ecosystem also gives you lots of options when it comes to building the signaling solutions. You have modules such as AIO, HTTP and WebSockets, which are very handy. And you also have lots of options if you want to do things like image processing, or even machine learning with projects such as OpenCV and TensorFlow. It's easy to feed these media streams or the frames of these media streams into these projects. AIO RTC comes with a growing collection of examples built right into it. So on the left here, what we have is streaming BigBug Bunny from an MP4 file into the AppRTC demo website. So this is something you can do with zero lines of code. There's an example for that built in. And on the right-hand side is an example where the browser is talking to a Python-based server which handles both signaling and media, which applies some real-time processing on the video frames and sends them back to you, in this case, with a cartoon kind of effect. What are some of the use cases for AIO RTC? On the data channels side, you can use data channels, for example, to communicate with embedded devices. Or you can have some more esoteric use cases such as running a VPN over data channels and so benefiting from the firewall punching features of WebRTC. There's an example for that on GitHub. There's also quite a wide range of applications which involve media processing or maybe machine learning. You can do things like real-time feature extraction and recognition on video streams. I've had some users report they wanted to use AIO RTC for a central server which would record video streams coming from mobile devices. Or if you want, you can also build your own solution to security, access your home surveillance cameras on the go from your mobile device. Obviously, this is Python, so one of the strengths of Python is how expressive the language is and how quickly you can prototype solutions using Python. The syntax will be very familiar for anyone who has used WebRTC. You're going to find your usual RTC peer connection. Thanks to Python's support for Async Await, you just do your create offers and set local descriptions as usual. What's unusual is that you have some higher-level objects such as a media player and a media recorder which allow you to either read or write your media streams. Now, what if you're operating in a really constrained environment and Python's not even an option for you? Do you have any good solutions for that? Perhaps I do. So, RRTC is another alternative implementation, but it only supports data channels, so it's a little bit specialized. It is intended to be resource-friendly, so you can use it in embedded devices as well. It does use two libraries underneath, which is RE and UserSTDP, and I originally created it in 2016 for testing purposes as well. So, my former professor Micheal Tixen wanted to have a tool to showcase data channels and test and improve the data channel implementations without having to work with or untangle all the existing browser's implementations such as the one used by Chrome or Firefox. Okay, sure. C was a requirement, so I wrote it in C. As I said, we used it to test a couple of things in the data channel implementations, it was being used to patch the EOR problem, which is also known as message integrity violation, and then we backported that to Firefox, and there were a couple of other improvements where we did the same thing, so we also tested throughput of the data channels and then backported the necessary changes to Firefox. So, of course, since it's now an existing implementation, you can also use it for your own use cases such as applications and services. So, one of the things that I have seen that seem to be of interest is integrating it into an existing torrent library to implement web torrent. Another example I've seen is some people seem to be interested in doing peer-assisted CDNs, which are just CDNs which are being, well, where you would use the peak load by sending data via peer-to-peer, and you could use raw to see for that, but then there are, of course, also the embedded use cases such as using it for, well, we've used it for RC toys, so we've made an example where you can control the Lego Mindstorms robot with it, and it worked fine, but also IoT use cases are in it as well if there is a little bit of power on the device. So, yeah. For example, exterior, interior, illumination, and, yeah. Furthermore, you can integrate it into an existing robot. You see implementation. If you don't have a data channel stack yet, this might be interesting. So, if you have eyes and details, for example, in a selective forwarding unit, then you can integrate raw to see into it so you can use data channels as well. This is one of the demos we wrote. So, this is on the right side. We see a browser that just opens multiple terminals and, yeah, it just accesses the local terminal on that device by using raw to see, which is kind of fun since you can punch through the net and, yeah, just access your device without having to forward any ports. So, in the process of producing these two alternative WebRTC implementations, we encountered a number of common problems or had some thoughts about this and which we wanted to share with you. First of all, what are the challenges if you decide that you want to spin your own WebRTC implementation? Personally, the first problem was finding the relevant documentation because documentation is spread out across, let's say, two different worlds, the IETF world and the W3C world, and so you have to hunt down all the RFCs or possibly even draft RFCs and then refer back to the W3C specs for WebRTC and trying to wrap your mind around all this is quite challenging. So, I think we're maybe missing a sort of single entry point which would refer to these different documents and give us a better overview of all the relevant specifications. The second challenge is that when implementing WebRTC stack is that this is a deep stack which involves a number of layers and it's only getting bigger. You already have to, you already need to deal with things like network connectivity at the ICE level, then through encryption using key derivation using DTLS, RTP and SRTP and RTCP for the media streams and on the data channel side, well, you need an SCTP stack. Now, unfortunately for me, pretty much none of these building blocks existed in Python. They do now. You can, and you, we agree that it's a good idea to spin these out into reusable modules so that if someone else wants to take a different approach on implementing this WebRTC stack, well, at least they have some of the basic building blocks to go on. A similar point is how to structure your code. If you've manipulated WebRTC, let's say with a browser perspective, you're used to this very central RTC peer connection object and from there it's kind of not very clear how you should structure your code. Luckily, ORTC, ObjectRTC, which is a sort of different approach on the same project, does break down this kind of monolithic stack into some discrete objects and it provides some interesting guidance in how to structure your code. This was one of Leonard's tips early on when I started implementing AIO RTC and I'm very grateful for it. Another issue you may run into is that you may run into parts of the specs which are ambiguous or maybe downright wrong and so it's challenging to contribute back to the W3C as substantial contributions are only allowed for members. Now, there are a number of benefits to having alternative WebRTC implementations. The most obvious for me is that a standard only lives up to its name if it has multiple implementers, so in this sense, diversity is really a good thing. Also, well, these WebRTC implementations are a lot smaller than the WebRTC.org one and personally I find that a lot more fun to hack on and easier to integrate into custom projects. In the process of developing these WebRTC stacks, we shook out a number of bugs in the browsers and, well, for the friendlier browsers such as Firefox, it's been a pleasure to contribute to the development effort. And having alternative WebRTC implementation also helps give valuable feedback to the standardization process as you're able to prototype new features or possibly explore areas which were not originally envisioned in the WebRTC scope. Okay, so now that we've heard about the benefits of alternative implementations, so maybe there are areas where we can improve and I think there are. So one of the things is that browsers are still surprisingly far away from spec compliance. So if you want to get this changed, maybe it is the time to get involved. There's another problem that is at the data channel lobby I think is underrepresented in the specification process, which also brings me to the next point, which is how can we make the specification process maybe more transparent, maybe more visible so we can involve developers and users to provide direct feedback instead of, you know, posting it on Stack Overflow and then be done with it. Last but not least, from the feedback that we get is that WebRTC is still misunderstood by developers, so some of them think it's mainly useful client-to-server while it is actually mostly peer-to-peer. There, of course, use case-away can use it for client-to-server, it tends to be peer-to-peer. And the purpose of the whole signaling process and how it works, how the mechanics work in the WebRTC stack, they are widely misunderstood. So we think the documentation could maybe be improved, for example, in the Mozilla developer network, but maybe we should also post new blog posts that update the existing ones because there are quite a lot that are outdated. So with that, we thank you for listening. We have listed some further alternative WebRTC implementations there if you want to look at them. Do you have any questions if we have some time left? In the meantime, we're going to let this figure sink in, which is kind of the ratio between the number of lines of code in WebRTC.org and in AIO RTC. So for AIO RTC, what are you using for congestion control for the media side? Congestion control on the media side. At the moment where congestion control is implemented for data channels, that's part of the STP spec. There is no congestion control, per se, on the media side. There is a receiver-estimated maximum bandwidth and so the video codec does respond to this and will adjust its bandwidth. For audio, for instance, there's absolutely no provision for this. Either you have the bandwidth or you don't and communication breaks down. I hope I answered the question. Last question, a quick one. Otherwise, you can meet the guys around here. I completely agree on documentation. It sucks and the Mozilla developer network is just wrong, 99% of the time, when it comes to WebRTC, which is a real shame. It's not really a question at all. I'm completely agreeing with you on that front. I just want to talk about that 0.5 number. There's an awful lot of AV stuff in that SDK codebase and it's difficult to talk about a total number of lines when we're talking about transport and encryption and AV. I don't want people to think that, oh, why is the Google one really, really big and yours is really, really small? Definitely, this is just very... This is kind of being a bit cheeky. Still, within the 6K, the 6K lines of code that I mentioned, you do have a full ICE implementation support for DTLS, and all the way up to a receiver-estimated maximum bandwidth. Still, I think that somewhere, for sure, AI or RTC does more than 0.5% of the features. I'm not claiming full feature parity, but still, there's a massive Pareto effect going on here. And concerning documentation, on MDN, you can contribute things such as what browsers support which features. We've both done some contributions to this effect, and I encourage you to do the same. Okay, thank you.