 Good afternoon everyone. I hope you're not sleeping on your lunch anymore So I'm Remy as I was introduced and Well, I'm not gonna talk about DLC 4.0 that much because France talked about that a few hours ago, but I'm just gonna Talk mostly about actually more about Rayland and VLC really Yes, this is this is just my point of view. Please don't quote me as Representing my employer because I do not represent my employer. Thanks So quick introduction about myself. I joined video land so the VLC project as a student in 2003 So that's 16 years ago. Well, okay 15 years ago and I Stayed on the VLC project as a hobbyist after after I graduated I I spent a whole long quite a few years working for local in your killing expanse, but of course that didn't Turn out very well as you might remember and nowadays I work for who are research as a software engineer and Have become the largest At least that in terms of source code and the largest contributor in the VLC project history But let's get on to with with the actual content This is a bit of a Like chronology or milestone of VLC a supported wayland and nowhere in a way. Oh, I think open. I think it's a good example of all Open source project that picking up on where I land in general. I mean we of course we will see variation but So in 2010, I think we were already starting to hear about like this fancy new window in management system that would supposedly be coming from I think Intel research and Well looked at it and I was like, well, okay, so we need to support this EGL thing and EGL is the replacement for GLX and it's basically how you bind your That's how you bind the windows when you're on your operating system to an open GL context So you can then draw a 3d content with open GL. Oh Render should I say? But didn't really go very very far back at back in the back in those days EGL really only only supported X11 on Linux. So you couldn't really do wayland with it So it was kind of going nowhere and for four years Nothing really happened which got us to 2014 and Finally at that point we were able to have VLC draw some kind of image In wayland using using actually EGL 2015 which we're using MMFD So if you are familiar with Linux, that's kind of the more modern way to handle temporary memory segments Our share memory segments 2017 we finally got the first version of the XDG shell which is a Code word sort of for the name for the window management like the desktop window management in in in wayland then the stable version a year later and also last year we finally got input Input working which means that you can actually use your mouse and your keyboard with VLC on wayland And you also have the cursor working So hopefully this year we'll get it in a usable shape and it should be shipping in for the zero but this has been a really really slow as you can see incoming and Yet we still don't have screen capture Which is basically recording your recording your desktop so using VLC to record your Your desktop and and code it or stream it out. We don't have global hotkeys, which is stuff like multimedia keys or any custom Global key you might want to associate To VLC even if VLC is not the active application is not the focused application and The skin engine is also not supported in wayland And I don't know if that's more like a lack of engineering resources And I'm not sure if really nobody's anybody is still using the VLC skin engine anymore So maybe we don't really care But we'll see why we're not doing the screen capture and the global of keys later in the talk So in a nutshell what is very land for those who don't know So as as mentioned in the intro it was basically a successor for X11 windowing subsystem, which is which is our pretty much all Linux and BSD and Well, mostly nooks and BSD and our unique systems in general except macOS use for drawing Applications well for drawing at all really in graphic mode unless you're in plain text You're probably using X11 and like as a wayland like like X11 is is is a client server protocol Each app each applications with us at your browser your VLC Your word processor is a client and you can have multiple clients at a time each application that is currently running will be a client And you have one server which is the display or the compositor in wayland terminology, which is Basically one graphic card with one or more monitors connected to it It's it's an inter-process communication protocol it runs over a socket usually a unique socket X11 could also run on IP sockets, but But wayland Doesn't really it's really just a unique socket of the time Like also like X11 as a protocol definitions all the message formats and parameters are defined in an XML file Now in in X11 it was more like an afterthought and and something that the XCB projects added after long after X11 was already taken into use for wayland It has been like that from the beginning and that also means that in waylands case The specification is also part of the XML now It's just comments or text in the XML pros for a compute for human story It doesn't affect the computer, but but essentially the protocol specification Is also part of the protocol definition and it's in it's in the XML files Input management, especially keyboard handling is done with XKB common You might have seen that library named among the package names on your Ubuntu or DB and distro or whichever in this way you're using It's it's basically the library that passes key maps Keymap configuration files for the different languages and keyboard keyboard layout that you might have It's actually a rather complex thing because you have especially well if you don't use Latin as your alphabet as your main as your main alphabet and put Writing system you probably have a switch and Support different mappings for different keys depending on key states, so it gets quite complicated Wayland actually shares the same input system as X11 and both of them are passing those files in client side Using this XKB common libraries and the X actually refers to X11 in this case, but it is used for wayland The key differences between wayland and X11 wayland is a very tiny core protocol The baseline protocol in wayland is extremely small and simple Whereas X11 has thousands of different message messages that it can send Wayland is totally geared toward shared memory. It doesn't support Wayland itself does not support remote desktop if you want to do remote desktop with wayland you have to Convert it into a different protocol typically VNC or you could use a remote desktop if you are more Microsoft minded So of course you can use wayland as a client or the server side of a or the remote and the local side of a Remote desktop session that you can't use it as a as a protocol that's around over the over the network unlike X11 But that's just because I mean most most cases you do your application is running locally and for performance reason You want to have shared memory and of course shared memory does not work across the network by definition And another big big difference and that does affect Software developers a lot including myself is that wayland has very strict access control So a single client or a single application a single process can only manage its own Objects within the protocol and so it's on windows or surfaces as they are called You cannot See or you cannot definitely cannot modify in you can't even see with the wind Enumerate the windows of those are always our clients. This is a security feature by design of the protocol It's meant to be to enable Sandbox applications running directly Talking directly to the to the compositor without to the display server without any kind of Proxying of course this does cause problems such as as I was mentioning earlier You can't do screen capture and you can do input grabbing so you can do a global hotkeys and that's Because obviously well if you can screen capture you can see what other applications are doing and this is goes against the security model Another side effect is that you can't do nested nested windows between different processes which can sometimes be a problem Less nowadays, but it was it would have been a huge problem for like web plugins For instance, there are actually extensions where you can explicitly enable this on a windows basis though But I won't cover that today So anyway, if you want to put your application to wayland and this is basically the thought process that I've been into like four five years ago Well either you open the API documentation or you just all your feeling courageous and you directly open the header file and You'll see that basically all the functions except for this one W display connect Require another existing wayland object So you're like probably I need to start with this one and that would be a very good guess you need to start with this function to create a connection to the display server and Then you keep browsing and you're like what can I do now that I have this dpy this one now that I've connected to this display what can I do with it and That's where it gets tricky because you're like you realize eventually that basically there's nothing you can do other than call this display get register function a Get registry I think I don't remember anyway, and you're like, okay, but now what if you're used to X 11 you can create windows and and cursors and and All sorts of like images or sort of things in real and you can't You just get this registry object and you're like, I don't know what to do with this I don't know how do I create my surfaces or do I do anything? So it turns out in real and there's no There's no responses to requests Everything is based on Events everything coming from the server back to the to the client application is based on events and so what the registry does is Is basically provide a list of the supported protocol extensions and baseline protocol features and What you need to do is get the registry and then the server will then eventually send you events that oh I support this extension. I support that extension I support this standard feature and so on and so forth and every time you find one of amongst that list of Protocol extensions one that you actually need or might want to use you need to bind that extension to an object and There they have they have some kind of weird naming convention. So W L is for the core protocol, which is supported by everything WP is Basically the same thing but defined later. I think they just had some It's not very clear what the difference is really between them ex-dg's for more recent extensions defined by the cross cross desktop working group So mostly Katie and you know, I'm people actually So as I was saying everything happens by events and another thing that is kind of Different from what you might be used to when you program Network or IPC protocol is that that is a standard within library It doesn't give you a queue of inbound messages Like you say get next message and then you start passing it and then have a big switch statement it doesn't work like that instead it it expect you to register your callbacks for each types of message for each object and It actually time at runtime generates a type safe function call Even so it see and you normally don't have runtime call generation in C it actually does generate a type specific function calls at runtime and for this it uses FFI which is library for doing that kind of That kind of things if you're interested in the details you can you can look at FFI on the internet or lib FFI I just mentioned that You really need to be careful with version numbering because the version number of the protocol is basically what Specifies which events and what parameters you support and if you mismatch them the runtime calls will not match between What you have written in the source code and what the runtime is actually expecting and then you'll crash immediately But okay is the main purpose of way London before it X11 was to of course show something on the screen so All we used to draw and know you cannot draw anymore. So So Well and doesn't have any server side painting which unlike X11 you cannot beat pixels you cannot draw lines or Circles or triangles like you could do in X11 and there's also no server side fonts Which is something that nobody was really using but X11 or supporting You cannot do hardware overlays And if you don't know what a hardware overlay is is because you probably didn't use them when they still existed nowadays You don't see them anymore. So you don't need to worry about them bit more Bozering maybe is that there is no modern so-called modern 2d rendering API in the wayland Which would be similar to see rendering X So Brenda is basically just drawing rectangles and putting them on on top of each other and scaling them and rotating them Or you can draw with wayland where you can use EGL as I mentioned at the beginning Originally wayland was meant to be used with the open GLES subset But by now It's been of course actually to support desktop open GL as well You can also use Vulkan if you want to do the really modern way So and then if you just want to share plain simple RGB pixel bitmaps or big maps You can use shared memory and just have 32-bit RGB But then you need to do the entire rendering Down to the last pixel in software and that's probably gonna be slow. So you typically don't want to do this You can also do DRM Direct rendering manager not not digital management Which is the way that Linux manages low-level buffer access for GPU But typically it's something that the low-level drivers do for you. So you don't need to worry about it You can still sort of emulate render functionality So you have the viewport extension which supports scaling in hardware So you can you don't have to scale in software your your your Your PIX map and it can also orient things so it can rotate at square angles or flip And you have the subcompositor which enables Blending multiple Surfaces and there's uncalled sub surfaces so you can put surfaces on top of each other Unconditioned there from the same client Another surprising thing that you will definitely hit if you do well and development and you port some existing application Is the way that window size is handled And first windows are called surfaces in real and but this is our windows The surface size or the window size on where land is defined by the content that you are that you have last attached So the last drawn on to the Attaches the correct word to a surface. So you say I have I have the surface here It's my window for my application and this is the content. I want to show at the next Rendering until further notice and this the size of that content will define the size of the window on screen It's not it's it's completely the other way around of what it what used to be in in windows are X11 where The system tells you now your window is 100 pixels by 200 pixels And if you draw outside of boundaries, I'm just gonna not show it and screw you You just have to follow that size in real and is the other way because they want to have a perfect big pixel perfect rendering The client says now I have a surface ready of that this size. You have to show it at this size and And so it works in such a way that for instance if you drag and drop a window Or if the system wants to resize a window for some reason it will ask the application Hey, it would be nice if you could resize a window to a different size But you don't have to do it and when you do it you have to then be careful about the ordering of the event because you have to first I prepare the surface is a new size then acknowledge the new size and Immediately provide a new size. Otherwise, they're gonna have some Disynchronization between the size of the window and the content and your window is gonna move around and it's gonna look really ugly another thing to Keep in mind is that in Wayland there's no decorations by default or there's client side decorations. So means that The application is responsible for drawing the border and the title bar and all the decoration that that you typically have provided by the by the window manager and Well, this is a bit controversial There is now an extension, but it's only since less than a year After much much flame words and discussions in the communities as there actually is a way now to ask a server to provide Server side decoration if you want them, but some servers still don't support it and are and some developers are adamant that they will not support this but this is something to consider so normally your UI frameworks or cute GTK or whichever you're using will draw the decorations for you, but but if you don't have a UI framework, then you will have to draw the decorations or then hope that the window manager can't provide them for you Because this I'm not gonna go through and I think done So if there's any questions Nope Well, thanks, then any questions now. Yes Yeah earlier Yeah earlier I was mentioning about how do you send wayland content over network and I mean earlier When you were saying that there isn't any core support so the key the key selling point for wayland There's two key selling points for wayland The first one and it's a very bad Selling point in my opinion as an application developer is that they wanted to get rid the developers wanted to get rid of the x11 legacy which had become untractable and It might be a valid point if you're an x11 developer, of course as an application developer like myself I don't really care. It's their problem. Well, it's not my problem, but The other argument and this is the more interesting one is is is pixel perfect rendering So in wayland everything is synchronized. You don't have this Effect I find I mean you must have seen it on especially on slow computers when you take the window and you resize it with the mouse Very fast and the decoration moves But you see the old content and it's not as I said so in wayland This does not happen because because as I was mentioning with the size thing The window manager will tell your application now your size should change Please change it and then the application will tell when it's ready to change to when it has updated the size And so everything would be synchronized nicely And that is the key selling point and this is why wayland is basically a compositor and not This place ever in a traditional sense of x11 is now regarding regarding Remote desktop There was a bit of a controversy because it's not supported in the protocol but of course you can still encapsulate it in some other protocol and Because you have to sell perfect rendering and because you have this more optimized Communications protocol than x11 was and you have less latency issues and then you might have had with x11 But to be honest if you have latency issues with x11 Just switching to xcb from xlib at the application level Can already help a lot in terms of reducing the number of round trips that you need to do for rendering purpose But there also I think I'm answering the issue here and there is actually an extension for Wayland which which does actually provide vnc style Protocol over network and that's that brings the efficiency and the solution Really for the running applications on Linux over network Hopefully with a similar level of efficiency as the Microsoft RDP protocol that works on the widgets Any further questions that Okay, let's say thank Remy for his talk