 Good morning everybody, welcome to Room 2. Our first talk in this room is QDG's QDG streaming by Regisus Ferdinandis. I'm sorry, I didn't come out currently, but... This talk is not completely related with QDG, I think. Okay. I'm going to talk about QDG streaming, something that I have been working for the last 6 months already. I will introduce myself. I was working in security companies, security-oriented companies, QDG backup, for example, gaming companies, and also now I am part of the QDG company working as the 3-tormentator of the QDG network, and I'm sure that I will be the maintainer as well to this point. If you have been living in a cave or sleeping for the last 5 years, maybe you don't know what's WebGL, probably you don't know. A lot of fancy stuff is WebGL. There were some tries in the past to get something to have some kind of 3D wall in the web, in the past, the PRML, many other companies like, for example, Adobe with Flash tried to create tools to allow developers to create 3D stuff in the browser, and the Chrome OS team came to create the standard that is called WebGL. WebGL comes directly as not a fork, but it was inspired by OpenGL ES2. This is something that allows us to have a plugin that allows to convert our OpenGL ES2 that we have in the devices, or even desktop computers or in phones to some kind of WebGL. I will explain how to do that in the following slides. There is a new version of WebGL, as you can expect WebGL 2, but it's not supported yet. What's WebGL is different. So, as I said, it's a way to bring the OpenGL ES2 context to the web. So, we can have an application running Qt Quick in our QML, fancy QML, and support it with the browser over the network, or even in local host. Local host is going to work better than you can expect. For the Internet, the latency is really big, so the performance is not good, but the technology gets even better. We can have applications being streaming in real time, as you can see. It enables streaming of Qt applications using OpenGL ES2. Use cases of this. We can have remote application access, for example, This was done in the past with other solutions like VPN for working with SSH on Windows with a RDP protocol, many others, an X, for example. This is a way to allow, to have an easy way to balance applications or share applications with others. For example, we can have an embedded device, which is completely pathless on the screen, and we can connect to this device running a Qt application, and control it with our phone, with touch events, or with our computer using the keyboard or the mouse, everything. Or add some presentations, for example, and many other use cases that will come in the future. Now, I'm going to be more detailed about the implementation. This plugin is a Qt platform application of Qt. It contains a minimal web server to serve, as you can expect, some scripts, web pages, even images, everything. It also contains a web server, because this is from a web server. We need to have user interaction, so we need to send events back to the application from the browser. We need to have some kind of interaction, which is provided by our web server. Also, one of the important parts is JavaScript, as I will explain this later, but of course, it's part of the browser, so JavaScript is mandatory here. And of course, we're here to paint everything on the screen. What's Qt, or Qt platform, as a function? It's a set of interfaces to customize the behavior of the Qt applications in different operating systems. We can write the same code in a Qt application, or even in an internal Qt code, and customize the different behaviors in different operating systems using these kinds of plugins. For example, in Windows, we are using a Windows plugin that gives us how to create a window, how to interact with the system, and many other things. We are using the XTB plugin, and this is about this new plugin called GL plugin. Also, one of the important things that these Qt plugins are doing is how to resolve the OpenGL function pointers. This is important because the way this works, is that we need to hook these functions and overwrite them to have that customized behavior. Instead of rendering it directly to the screen, we are rendering to something that we are going to see later. About the web server, it's a really tiny web server. It uses Qt. We're all hitting two days, but it works. It has to serve some webs and nothing else. It opens the connection with the web server. In the future, if we can improve it, we can do more fancy stuff. For example, sharing the clipboard with the browser or something like this. It's a temporary solution. It's still working progress. This was something that I needed to enable the rest of the things. WebSocket Server. As I said, everything needs to be sent to the browser, and also the browser needs to send some feedback to the application. Basically, this is done with WebSocket. Every client connected to the WebSocket Server, which is in the application and also in the client. I use it to send GLDS to calls in a binary format. As I said, if the user clicks somewhere in the screen, this WebSocket will emit everything via this WebSocket. If we are calling a GLDS command that needs a response from the browser, in this case the browser, the WebSocket will read the response to the application. This part is the most tricky. It receives the GLDS to calls in the binary format. It's binary because it's more compact, less data over the network. The JavaScript file contains functionality to serialize this GLDS to calls and convert into something that the browser can understand. In this case, it works GLDS. And also install some user handlers to be able to receive the clicks, touch events, or resizes for something. WebGL, basically, is showing 3D stuff in the screen. If you have been reading my blog post, one of the most popular tests was the particles. So now I'm going to show you this demo in Red Hat. Basically, the demo's here. This is a QML scene application running a presentation, which contains a particular slide. And if I move the browser, because this is everything in the browser, we can see that we were using the first demo from the beginning. Now it pressed F5 without the clicker. It's supposed that when you... This works in the Funnest Web, so I wanted to show you how the QML scene is sent to the browser. Here is the presentation. Okay, well, so I can go back and forth. But what I'm going to do is just reload from here. And the presentation comes back to my laptop browser. So what's supported right now? The main use case for this is to stream huge quick applications, because some people were doing a lot of effort to optimize the Qt scene graph for Qt Queen. So it works really perfectly, I can't say. So it will be optimized, and it's not possible to do it better than we say. Or if we have an OpenGL application, we can simply stream all the calls if we are using the driver as well. And we'll forward everything to the browser, because they're boring. In fact, we'll have time, maybe, we can show the typical to example I showed in this slide, running in the browser, maybe. In the past, I said that this is going to be able to allow us to have some kind of multi-user support in Qt applications, because depending on how I wrote the architecture in the past, everything was executed in the same process for every browser connected to the application. Instead of sharing the entire application, I was sharing a window. There were some problems of this. First of all, we need to create, to write or modify our applications to support this WebGL planning, receiving some special events that allow us to create an application for our browser, or for customizing it, or giving proper content. This was not discoverable, because since we need to share the input contents of the application with several windows instances that are focused at the same time, distributed a lot of problems and mess. Another problem was when OpenGL needs an answer for one of the calls, a very window stops updating, because we need to synchronize every one of the contents. OpenGL works in the way that we set a current context and just one is not a mess. This is going to be a problem, but there are some work-arounds. With these work-arounds, I think we can do it better. Even, for example, with Manning Group Security, something that always makes people... To support multiple users at the same time with displaying, that's not what you're doing. One of the things I'm doing, or I'm going to start with, is trying to decouple completely the HTTP server from the application. Instead of having an embedded HTTP server in the process, we are going to have a common HTTP server that is able to span new processes. Every process will contain their own socket file descriptor, so it will be able to communicate with the browser, but in a separate process. So if the process crashes, the server will continue accepting more requests. So just a process dedicated for a client will crash. Now I'm going to show you how to convert the function from GLES2 to WebGL. For example, we can... If they attach, say, Rust as an example, something I forgot to say hello to WebGL is that the API is quite similar to OpenGL ES2, but since it's based on JavaScript, they can have some kind of object-oriented paradigm. So this is the most important thing. I need to keep a map of IDs from OpenGL ES2 to an object in the map to be able to, for example, to attach, say, to pass to the original function with this program map, which is to construct the map, which maps everything. The way to override the functions is with a feature or a problem, depending on the point of view of JavaScript. Since there is no public paragraph or detected meaning, you can access everything, you can override everything, or you can override everything. So something I did is simply take the original WebGL function of the context, in this case, WebGL, and set a new function or variable in the GL context, starting with underscore, which is the original function. And I override the original function with my own function, which receives the parameters from the WebGL... Well, the serializer implementation for all the functions are there more, but I show just some of them, for example, which is a bit more complex, and it does some stuff to convert it. I know this is a crazy question, but there is no chance that this would ever work for widgets, right? In widgets, no. But we can... I don't know. I have no answer to this. If we rewrite widgets to be more compatible with OpenGL and draw everything in an efficient way with the GPU, yes, why not? You can... this is a real example. Right now there are some bugs, for example, in this... every frame free update, there are some flickering, so it's not perfect right now, but if we implement something that just calls OpenGL with our application, we can draw everything. So if we want to have a new widget system, we can do it. But yes, we need OpenGL. No, raster is not a good idea. Right, and there was an OpenGL-based paint engine that's going on. Yes, but I try to... it's not a perbinator. Right. Basically, I think it submits a lot of textures. Right now, something I want to say about the display is that I'm forcing every texture to have a limit of 512, which is the minimum I was able to support to try to reduce the size, because if not, people will start sending 2K textures and will take the image. So yes, the answer is no. It's not going to be supported. Right. There are other solutions, for example, you can create an application with good OpenGL, of course, and maybe draw your interface with there's an open source rate called I am going, which is really good. Maybe this is a solution. If you want to have a more better interface with trees, then this is... Right. No, but I'm thinking about... I just have a lot of people who have students from JetApps that are able to, you know, make it possible to use that on the web. This is not a solution for huge applications. It's just to show... 10 controls or something like that. It's not a typical complex application with three wheels. So, now we're going back to the good word. If a company wanted to have an application that users would use through the web with this, is it correct that there is one sense of the application running for every user? Right. So if you have a thousand users, you have a thousand instances of the application running on the same server, if you want to have a thousand users using this technology, right, they all have a library in front of them, but they are all using the different instances of the queue application on the server, right? We will... We will be limited by the operating system of how many processes you can spawn or even by descriptors you can have, because we need a shortcut, at least. Right, so at some point you might have some sort of... Yes, but I think it's the point that we apprehended that in certain space because you have a powerful reward server with a pretty good connection. I have a demo in my... in my awards edition in Oslo. But the... I'm not going to show anything of this because the latency is 120 milliseconds. So... But if... If I connect from my phone to my office, it's possible to work even in remote. Something that may also need to be refined. Let's see... How is the license? Is the source code for this topic? Yes. What is the license? You can check. OK. The link is probably going to be there. It's DPR3. DPR3? OK. Let me show you the ocean. OK. Any other questions? Yeah. All the code to go from the ES2 code to then the binary format to then to the JavaScript on the model that's all encrypted? Yes. Do you plan to generate that somehow through recent maintenance? Not only. Simply watch. What do you want to do with this? Well, because it's a lot of code that you get to maintain, right? I'm just wondering if I want to... want to support seeing WebGA2 and DPR3, right? I'm just wondering if you can throw it away and put stuff in scratch. I'm just wondering if you have a mind of trying to put something in that, if you have something on your right out to do that? Not yet. OK. But it's working for us. Right now the repository has four comics and two on the queue. So I really want people to contribute with this. Do you have time for one more, please? Next stages.