 Okay, thanks for the introduction. So I'm Lorenzo. I'm from Napoli in the south of Italy Which means that pretty much the only thing that you really need to know about me is that this is a big no Okay Now that now that we have settled this for good once and for all Let's let's start with the presentation. So I don't know how many of you actually know about John was already This is going to be just a very quick recap about what it is Basically, it was conceived as a general purpose WebRTC server and I'll try to explain by what we mean by general purpose in this sense it's of course all open source there are demos and documentation available there and The idea was that we would write a core that will be responsible for all the webRTC stack So Stephen just explained what how webRTC works what it is basically these webRTC stack allows Janus to act as as a browser in the communication in that peer-to-peer communication that you saw before so These were vertices that was responsible for all eyes details all the protocols that that relates to webRTC And then we have different models to take care of the rest So we do have a Janus API so a way to communicate with Janus to control what it needs to do and how and this is Available different models can implement different protocol transports for this but more importantly We have all the media logic implemented as different plugins as well Which means that this way you can just write a new plug-in to have Janus behave in a different way So handle the peer connections that you created with the core in a different way So by default by the stock plugins allow you to have an SFU plug-in something that allows you to do some Basic video conferencing. Let's say an audio mixer ways to talk to the SIP world way to talk to RTSP cameras or whatever So all the logic belongs there So if there is a new need that you have that is not covered by the existing plugins You just need to write a new plug-in to handle media room Added to Janus and it will it will theoretically work The only caveat here is that Janus is written in C which means that you if you want to write a new plug-in You typically have to write it in C as well, which may or may not be an issue for you I mean for some it might be because you You need to know how to code in C and make sure you're not screwing up and causing issues of any sort and Just to give you an idea of how the plug-in API looks and this this is important Just because I'll get back to it in a few slides when we'll address the the purpose of this talk here Is that typically a plug-in has to implement a set of callbacks in order to talk to the core because of course if you Want to be plugged as a model to to Janus as a core application you have to respect some some APIs in order to communicate with it and typically it's just let's say some some ways to To register at the at the core to let's say this is my package name This is my version a couple of meters to initialize the plug-in and to destroy it and typically in it is called Whenever your plug-in is loaded which happens at startup So Janus starts starts to load the plug-ins You will get notified whenever your plug-in starts and destroy instead is called when the when Janus is shutting down so that you can keep Track of the life cycle of Janus itself that way some versioning information as we said but most importantly some ways to Handle users so to interact with sessions because typically the Janus core will notify you anytime that the user attaches to your plug-in We call this attach as a meeting so anytime that the user Joins your plug-in you are notified about this so that you can keep start keeping track of this user You can start receiving messages from the user You are notified anytime appear connections for this user on this specific connection came up or came down and of course most Important of them all you are notified anytime you have incoming data from that user So if the browser is sending RTP packets via the web RTC peer connection These RTP packets get to you in the plug-in so that you can then decide what you want to do with it Which may be I'm going to send this to all of the other participants in a conference or I'm going to decode it added to a mix I'm going to send it to a CPUs or whatever same thing for RTC RTP statistics and data channel messages as well because you may want to handle Just that a channel so that a channel sink conjunction with media and in general This allows you to track all the life cycle of a specific peer connection for a user, which is quite useful and Finally of course the core itself also exposes an API by which plugins can then start sending info back to the user Stuff like ways to send messages to a user via the Janus API And this is important because each plug-in has its own API You can consider each plug-in in Janus as a separate application living within Janus itself Meaning that each plug-in can actually implement their own messaging that then gets transported over the Janus API And you can do that via I'll show a couple of sequence diagrams to that show how these works and at the same time you can also Tell Janus when you have an RTP packet to send back to the user when you have statistics to send a data channel message to send And so these two set of API's allow a plug-in to both receive and send media at any time or a peer connection Which gives you complete control over what peer connection will transport for within the context of a session And then you have some additional meetings Let's say for close up your connection because you don't want it anymore and stuff like this and just to give you an idea of how this works from from a Sequence darn perspective the idea is that the user decides to attach to a specific plug-in and typically The user just specifies the plug-in name space, which is the unique identifier for a plug-in The Janus core knows that the plug-in is registered somewhere and so uses the create session callback to add the new user to the Plug-in then it's up to the plug-in. It's up to the it's a responsibility of the plug-in itself to to take care of the of this user And and do all what we've seen before so keeping track of the user Maybe create a new structure for this user to have to as a conference participants or whatever And then anytime that the user sends a message via via the browser or whatever meant for that specific plug-in the the Janus core will strip everything else and will just give the The message itself to the plug-in so that this can be processed somehow and The way that this messaging works is that Typically where plugins can handle messages in two different ways in a synchronous or an asynchronous Approach when you handle messages Synchronously then you just send a response right away to that callback that we've seen before so you receive the message You process it you send a response back and this can be useful for instance in like you will Like when I request these for give me a list of conferences that these plug-in handles and stuff like this in this case It's pretty much sinkers as you see but more often than not plug plugins actually communicate asynchronously instead which means that users send the message the core Passes the message to the plug-in the plug-in just sends an act back And then later on will send you one or more events notifications to take care of this The asynchronous messaging is particularly important because it also takes care of the WebRTC negotiation by itself So you Steven has anticipated how SDP offers and answers are involved plug-ins are responsible to Prepare that those are SDP offers and answers depending on who originates the session and this is all done in an asynchronous way So maybe a user sends an offer it gets to the plug-in the core will strip all the webRTC stuff It will just give you a bare-boned SDP the the plug-in prepares enough for an answer and sends it back so same thing when the plug-in is actually offering something and Finally of course me as we've seen media will be just bridged by the general score So the users send something via secure RTP the plug-in will strip the security part And the plug-ins will have access to the unencrypted traffic for for their needs As we've seen as I anticipated before all of these plug-ins and meters need to be implemented See because that's how see Janus was conceived. So Janus loads a shared model and expects See interface to communicate with but if we think about it We don't really need the whole plug-in to be in see if as long as the interfaces are Something that looks like see and the code is happy then we can implement all the logic in whatever language we want if we then implement the hooks in Somehow so that we have some stubs that then delegate the all the logic to something else which In principle, it's exactly what we did with the Lua plug-in Which at the moment is pull request that is available on our Janus repository I also wrote a small tutorial about how to write a Lua script from scratch to use these I'll go a bit on this later on and the basics are that Typically there are a few things that we need to consider so when for this Lua plug-in we basically decided to of course It's still a see plug-in because we need a see interface But then these plug-in loads a user provided Lua script when you initialize it So you say open this Lua script which can be maybe different scripts for different applications Then all the plug-in callbacks are implemented in see but they then call the Lua function so that the Lua script can handle those requests and we've seen and at the same time the the this plug-in also implements all the core methods that should be needed to communicate with the With the core and exposes them as Lua functions so that when I call push event from a Lua script The C plug-in will call will transform them in something that the core will be able to digest And in order to track user sensations in a way that both the C and Lua code can identify We chose just for a for simplicity to use a unique numeric ID to do that and In principle, this is exactly what we did so some kind of a C to Lua proxy communication Even though there are some problems that we needed to address that I'll talk about in a minute Which means that if you look at the date that API that we've seen before This is the C version of these callbacks that the plug-in implements because the core needs them But they are in turn then all passed to Lua to Lua functions on the other end The only two important ones in this case are in it and destroy because for the versioning information the Lua plug-in can answer It's in himself, but if you want to override that information as well You can basically and more importantly for all the Session management we have alternatives for that as well So whenever a user attaches with there will be a create session callback called on the Lua script and so on with the only Exceptions of incoming RTP RTCP and data which you might implement But you probably will not want to and typically you will not want to as I explained in a minute And for of course for sending his stuff back. It's pretty much the same thing So the Lua script will call push event or and session or whatever this will be Intercepted by the C plug-in via this Lua C communication and and it will be sent by the core to the core by the Lua to the Lua plug-in itself Which means that very simply this plug-in then acts as an intermediate layer between the C word and the Lua script So anytime something comes from one direction it translates it to to the other part and and vice-versa typically The there are there were a couple of issues that we had to implement you to take care of the most importantly We've seen how a synchronous event or more very important in the plug-ins communication because that's also how negotiations happens and The problem was that the Lua is typically Single-tread I mean they have a concept of threads, but they are not really threads They are mostly code coroutines typically and besides the access to the Lua state Which means the Lua script that we loaded that access the we might call it the virtual machine for the Lua Script that we are handling is not tread safe So we cannot access it from multiple threads without causing trouble basically So we had to first of all lock that to a state anytime that we wanted to access it and most importantly We had to implement some sort of a C scheduler that we could implement to take advantage of any time that we wanted to have some kind Of a synchronous communication So anytime the Lua script wants to have a message handled asynchronously It will basically just say create a core routine Keep it in a task list and then notify the scheduler in C to say I will need these to be handled in a core routine later on as soon as there is time for that Which is what we implemented via a meter that is called pox scheduler So the Lua plugin will just call this meter. This meter will handle a thread internally that then keeps keeps a queue in order to then Call the core routine as if as if it were a generic Lua function later on and you can see it Pretty much as exemplified here. I'm not going I will not go too much into the details be due to lack of time, but we you can ask for more information later if you're interested in this and And this is a very simple example of how you can do that. So You have a message you want to add you want this to be handled as synchronously you you add it to a To your own nash map or whatever you call pox scheduler And then a thread will wake up in the C code to then invoke Later on the be a resume scheduler the the task list so that you can do it again We also have an alternative to let's say time callbacks on on your own On your own schedule rather than let's say called this meter that soon as it is possible and you can do that VIM meter that is called That is called the time callback which basically means you say call this callback with these arguments in five seconds For instance and typically the approach is exactly the same We have a thread back in the C code that takes care of these as soon as the time pass We invoke the function that you wanted to be invoked which makes for a for an easy way to implement this as synchronous behavior That we needed but more in this is actually more important than the rest We wanted to have also a way to handle RTP RTCP and data in a way that would not completely block the Lua state machine because As we've seen we have meters to handle incoming traffic So an incoming RTP meter to handle incoming RTP traffic for instance the let's say the The the normal way you think you be you could do this would mean Take this RTP packet and pass it to Lua exactly as it is and let Lua worry about it Which is actually not an option because as we've seen Lua is single-treaded It will completely kill performances if you try to have Lua take care of this delivery itself so the the way that we handled it was to basically Handled this in a in a different way and this was because for RTP and RTCP and that as well The traffic may actually be very frequent Which means that you may actually kill performance if you did it directly there, so We basically implemented some meters that allow you to configure the routing of the media depending on how media is flowing in and then Let the C code worry about actually Delivering these packets in a one or either direction and the concept was quite simple. So we chose to Add a couple of meters that are called ad recipient and remove recipient that basically Allow you to say for these specific users all media coming from this user send it to these other users instead And of course you can call these multiple times for the same user so that you can have a one-to-one Conversation or a one-to-many conversation if you wanted and then the C plug in takes care of all the rest So they overriding the RTP headers if needed Sending packets around as they are needed and so on without the Lua plug in needing to worry about anything at all So it just let's say turn some knobs and some bugs here and there to say This is where the media should flow and this is flexible enough to allow you to do any kind of Communication from one to one to actually a complete SFU if you wanted to do that And this is a very simple example where for instance Alice is starting to send media Just Bob is subscribed to it and then we receive a new meter that says send this media to Tom as well Which means that the L the package that Alice is sending will be sent to both eventually We can see a couple of examples here So for instance we in that poor request that I've shown before we have a couple of examples out of the box So one of the examples was we just replicated the exact echo test that we have implemented in C as the Lua plug-in itself Which means the in this case it is really easy We replicate the same API and then we just make sure that whatever the user is sending we send back to him Which as you may expect it just means that As soon as the webRTC connection goes up and we know it because we are notified about this by the C plug-in We just call another recipient meter for this session and we pass it to the same session itself So that we say any packet that is coming from these sessions and it back to the same session with the which is Kind of easy, but already shows the flexibility that you can do with it. But of course this is more Almost done. Okay, sorry. I'll fly. We don't deal at this light So we also have an SFU clone so you can there is also a small video conferencing application written in Lua That is kind of a kind of cool where you actually do this ad recipient stuff for one too many kind of thing But I also wrote a complete tutorial to write a plug-in from scratch using a video call example and you can see it On our blog we have a complete post over here that goes all over the basics So it starts from the very beginning and it will drive you in explaining all the steps that you need to do in order to do that I also Implemented a small demo at an asterisk on a few months ago that took advantage of that and I also wrote a Demo specifically for this session here. Unfortunately, there is no time for that But if you go over that link over there, there is a something that looks like the video call But it's actually a chat roulette. So I assume that you know pretty much how chat roulette's work So you start you are not connecting to anybody then somebody randomly may join in in this completely Unedited and altred pictures may see Then people may go away and then another match may be maybe coming for me Which I may not be really comfortable with so I may decide to drop the communication instead And all of these happens within the context of the same pure connection So I connect to the plug-in once and then I just send buttons to say I accept this match I reject this match and so on and the plug-in in the background decides who should receive my media and Dynamically so in this way. So and I'm almost done here. So what to do next I mean where there are some things missing so typically there are the most advanced and stuff that we have in Janus are not Available in the plug-in yet like simulcasting and stuff like this and besides these Lua plug-in is also based on a branch that is experiment experimental by itself because it's the reference counters branch But there are some things that we may want to do in the future like for instance I mentioned that we have a couple of other type of plugins like transports an event handler So it might make sense to start to think about Lua for those as well if it makes any sense But actually something that is more interesting to me start experimenting with other programming languages as well Because Lua may be just an opening door for for this We already have all the hooks in place to decide how to then communicate with a different language and have all the hooks and Stubs in the C code so it should be easy enough to let's say use something like duct tape or something else To communicate instead with other programming languages with javas like javascript or Python or whatever else I mean this is something that I'd like to experiment in the future and I hope this was interesting enough for you to play with and so if you want to start playing with these or maybe Also start implementing a new programming language by yourself. Just just play with it and let us know it will be to be really cool So that's all I don't know if we have time for questions or if you're done Okay, sorry about that