 Hello, my name is Maxim. I work as a browser engineer at EGALIA and in today's presentation I'm going to present you the project I've been working on for almost four years called Chromium on Wayland. In this presentation I would like to present to you EGALIA and talk about who we are and what we do, in case you haven't heard about us. Then I'll talk about the history of the project when it started, what problems we had, how it evolved, and what where we are now. Then I'll continue by presenting to you the ozone component, explain the idea behind that, how you can reuse it in your own project, and I'll talk a little bit how ozone-intermediated it is, aura, and how ozone is used. I will also present to you some core classes of ozone-wayland implementation using some diagrams. After that I'll talk about the limitations or better to say problems that we faced while we've been working on the project and also explain how we overcome them. Then I'll compare the performance of the ozone-wayland versus non-ozoan X11 on my Raspberry Pi 3 board. Initially I wanted to do that using the Renaissance Aircar M3 board but as I said everything, as I had everything ready for RPI, it was decided to use that instead. I will also talk a little bit about shipping of ozone-wayland and why we also work on migration of X11 to ozone. InGaulia is a free software consultancy with headquarters in Spain and developers all around the world. It was established in 2001 with two main goals, open innovation and plus development. At this moment we are about 90 people working from different parts of our planet and we are experts in various areas. For example we have teams who work on browsers and client-side web technologies like Chromium, Blink, WebKit including DPE and WebKit GTK Plus, Firefox, Server, Brave, and others. We also have a graphics team who is professional in graphics pipeline and rendering technologies, for example hardware-accelerated 3D APIs, Mesa, Vulkan drivers and more. Our next team is proficient at compilers and programming languages and contributes to JavaScript through TC39 and they also work on V8, JavaScript Core and other stuff. Our multimedia team has experience with Gstreamer, VA API and many other media solutions. We also work on embedded Linux device drivers, accessibility, virtualization and cloud. Igale contributes to many different places and is a member of different groups or contributes to these different groups. So a worldwide web consortium where we co-chair the area working group also Linux Foundation, Chronos groups, eCNA and others. On this map you can see where we are from. Igale and Sleet and work in almost every part of the world. But okay enough about Igale, let's start with the history of the project. The project was started in May 2016 by our fellow colleague Antonia Gomez. In the beginning it was mostly a research project with a requirement to analyze whether Intel's implementation will be suitable for the upstream and whether we could merge it to the Hormone's mainline repository with as least changes as possible. We contacted some UI core engineers at Google and we were told that a new Moose plus Ash project is being developed and it would be wise to start to work with that from the beginning. That project goal was to turn UI into a separate service and it was initially designed for Hormone's needs. We also realized that Intel's approach wouldn't be accepted as it was designed in such a way that well-on connection was established in the GPU process that is a subject to security problems and sandboxing. Over a course of development and research we had been refactoring the Moose and Ash project so that it could be reused for Linux. However in March 2017 we were told that we should hold on sending patches with changes for mass and it was decided to move downstream instead. In April 2018 when we went to Blinkon 9 in Sunnyvale we presented to Google our work in more detail and after some discussions it was decided to use direct aura integration. I'll explain the design of ozone and our integration later. Since that time we have started to work in upstream again we merged most of our patches that were related to ozone well and continued the development on the tip of the trunk. It's not really related to the ozone project now but in 2019 it was announced that Moose plus Ash was finally discontinued and from September 2012 ozone is part of the regular Chrome builds and can be enabled by the with the runtime flags you can see here on this slide. But what is ozone? Ozone is a platform structural layer beneath the aura window system that is used for low-level input and graphics and it is worth noting that aura is actually a well as I said windowing system for chromium that's internal component which is platform independent and you can treat it as GDK or anything like that. The goal of the project is to make porting to different platforms easy. First of all ozone is about interfaces not if devs. That is the differences between platforms are handled by platform specific objects instead of conditional compilation. All the implementation details are left to the platforms implementing different ozone interfaces. Secondly ozone offers flexible interfaces. This must offer only what chrome needs. This idea also excludes dependency on chrome components only chrome depends on ozone. The third point is that ability to choose back ends with runtime flags that means a single binary can be reused many times. For example in the ozone support for Linux at the moment it is possible to switch between well and X11 and headless back ends. On Chrome OS builds there are even more options DRM, X11, well and headless. And the last advantage of ozone is that it eases downstream development. Not all of the back ends end up in the upstream and the goal is to make life of the downstream maintainers as easy as possible. With ozone one can plug in their own back end for example GTKQT or whatsoever and use that without a fear of next replacing. So given that you are now familiar with what ozone is let's speak about the design of ozone how it interweaves with aura and how ozone well and is designed. Ozone lives both in the browser process and the GPU process side. This diagram represents the UI part of the ozone that lives in the browser process side. On the left side you can see the platform-dependent aura implementation for ozone and the right side you can see the ozone implementation itself. So desktop window tree host platform is our main class that inherits window tree host platform that holds the platform window and it has a one-to-one relationship and in other words if a new window needs to be created and new desktop window tree host platform is created as well. That class is responsible for forwarding various requests to platform windows that ozone implements. Further you can see a class called desktop window tree host Linux that is the class that subclasses the desktop window tree host platform and has some Linux specific APIs. As you can see there are X11 specific methods. That class also implements X11 extension delegate that is used by ozone X11. Most probably will remove them once all the X11 specific bits are moved to ozone. It's also worth noting that ozone platforms that other platforms such as Windows and Mac may also need to implement their own desktop window tree host inheriting the desktop window tree host platform of course when they start migration to ozone because of some differences in each of the platforms. And going back to the desktop window tree host platform you can see that it implements the platform window delegate as well. This delegate is the way how ozone can communicate back to Aura. For example it sends UI events notifies if the window got closed because of some compositor side actions and it also sends notifications about bounds changes and others. On the right side is ozone. As was said before it's based on public interfaces and all the implementation details are hidden and are not visible from the outside. The core class that is also singleton is ozone platform. It has methods that platform should implement. For example initialization of UI, GPU, creation of Windows and others. Here on the right side I presented two ozone implementations X11 and Wayland that provide some implementation details for both of the back ends. You can see that in Wayland we have a Wayland window that implements the platform window interface which is then used by Aura to communicate with ozone windows and you can also see the platform event dispatcher that Wayland window implements so that it can send out events processed by the event source in ozone and X11 basically does the same. So once Aura internalized ozone and created a platform window it can interact with ozone using the platform window interface. In its own turn ozone communicates back to Aura using platform window delegate interface. For example Aura can ask ozone to show the window maximize it, set bounds and so on. In response it can receive an activation changed on windows that changed and other calls. Let's talk about the design of the ozone Wayland in more detail. As was said before our main entry point to ozone is ozone platform Wayland that implements ozone platform. The ozone platform Wayland is responsible for creating all the core classes. For example Wayland windows, Wayland connection, input methods related classes and others. The Wayland connection is the core internal class. It handles a connection to Wayland and holds a Wayland display. It also handles Wayland seats and registry. Whenever Wayland is initialized Wayland connection receives globals from Wayland binds the interfaces and creates C++ wrappers for example Wayland keyboard Wayland pointer and Wayland touch. It also creates a Wayland event source that is the main processing unit for events. Whenever Wayland sends keyboard pointer or touch events they are forwarded to Wayland event source that translates them to chromium's internal type which is the UI event and it also decides what Wayland window should receive the event. Of course this decision is based on native capture that Chrome sends and also on the focus events. Wayland event source is also the owner of Wayland event watcher the class that is responsible for watching for file descriptors readiness coming from Wayland compositor so that events can be read from the pipe. Wayland connection also owns Wayland ZVP linux demo above, Wayland DRM and Wayland buffer manager host. The last class is responsible for managing Wayland buffers that can be backed by DMA buff or shared memory. I will explain how this works later. And the last important point is our design for surfaces. Here you can see our core class named Wayland window. Its fundamental piece is a Wayland surface that owns a BL surface object. In turn we have several types of windows that subclass Wayland window. For example Wayland top-level window that plays our role of a top-level surface and owns a shell surface wrapper. This wrapper can be any surface but as long as chromium supports XDG shell v5 and XDG shell stable now we only have support for that. But if you work on a downstream project and you use another type of shell you can implement the shell surface wrapper for your own shell. Next we have a Wayland pop-up for menu windows. It owns a shell pop-up wrapper. The shell pop-up wrapper is again an interface that your shell can implement. In the upstream it is XDG shell of course and XDG shell v5. And the last one is Wayland auxiliary window. It owns a BL subsurface and it's mainly used for tool tips and drag-and-drop arrows. So that's about the core design and let's speak about the problems we faced and how we tackle them. During the development of Ozone Wayland we faced two problems. The first problem was Wayland EGL. Given that Wayland connection lifts in the browser process and all the GPU related stuff happens of course in the GPU process we had a problem. BL EGL surface couldn't be created because BL surface wasn't accessible from another process. This is the fundamental difference between X11 and Wayland. In X11 you can pass a handle for the X window to another process and use that. In Wayland everything is different. Each client has a sort of sandbox connection and other clients cannot manipulate objects that belong to other clients. However it is still possible to use Wayland EGL when the in-process GPU flag has been passed. That is GPU runs in a thread inside the GPU process in this case. But we use Wayland EGL only if LibGBM is not available. Another problem is the tab dragging feature. It was of course possible to use the existing drag-and-drop protocol for that but with the limitation. And the problem was that the way the drag icon is set. For example whenever a user drags a tab that becomes a window Chromium must be able to continue showing the contents to the users. For that it must be possible to use to reuse the very same surface as was used by top-level window. However the protocol didn't allow that because the EL surface has already had a role. To fix the problem a protocol fixation was proposed. So let's talk about let's talk more about each of the limitations in detail. And the first one is the graphics pipeline. So we had to make a design that would make it possible to fit frame buffers with content in the GPU process site and tell Wayland which surface these buffers belong to in the browser process site. For that it was decided to use DMA buffers created with the help of libgbm library and of course DRM render nodes. And use surfaceless drawing that implied reusing of surface's EGL implementation that Chromium has had. We also used Mojo for inter-process communication and Mojo is actually a collection of runtime libraries providing a platform agnostic abstraction of common IPC primitives. Message ID, IDL format and bindings library with code generation for multiple target languages to facilitate convenient message passing across arbitrary inter and inter-process boundaries. So we also used, in addition to just Mojo, we also used associated interfaces of Mojo so that we can ensure the order of messages that was really important. That approach allows us to enable GPU member buffer support and use zero copy with Wayland. So some diagrams here are to help you to understand the design better. In the GPU process site you can see the Wayland Buffer Manager GPU that is owned by Ozone platform and yeah as I said we usually have two instances of that one living in the GPU process site and another living in the browser process site. If we have only one process and we run in the in-process GPU we have only one instance of Ozone platform which is natural. So the manager communicates with the Wayland Buffer Manager host that lives in the browser process site. You can see this object in blue. So communication happens via Mojo and we pass domain buff handles to that. Also buffer IDs and surface IDs for buffer management. The host manager communicates with Wayland then and tells it that it must attach a buffer X with IDY to that surface and another buffer to another surface and so on. The buffers cannot be attached to different surfaces as they belong only to a single surface. To ensure the correctness some sanity checks are done before submitting the buffers and attaches well basically attaching them to surfaces of course. So when Wayland Compositor releases a buffer after it was submitted Wayland Buffer Manager host sends on submission call back to Wayland Buffer Manager GPU via the Mojo Pi and the Buffer Manager GPU searches for the GPU surface that is that this submission belongs to and notifies it about that. Same happens for presentation but these presentation calls happen a bit later than submission and it is guaranteed that the submission for the specific buffer will not come. Sorry the presentation for the submission for the specific buffer will not come before the submission is actually done. It's also worth noting that the DMEA buffers are represented by GBM PIXMAP Wayland instances that implement native native PIXMAP. The GBM PIXMAP Wayland uses libgbm to create DMEA buff and then passes a DMEA buff handle to Wayland Buffer Manager GPU which then asks Wayland Buffer Manager host and as a result Wayland Compositor to create VL buffers via either Wayland ZVP-LinusDMM buff or Wayland DRM wrappers. Of course usage of Wayland ZVP-LinusDMM buff and Wayland DRM depends on availability of those and if both are supported by the environment, by the system, by the Compositor the first one is preferred. So if shared memory buffers are needed to be created as well then Wayland Shim is used. So second limitation we had a problem and it was impossible to use the existing surface as a drag icon for homonyms so called preview mode and the existing drag and drop protocol had limited functionality. To fix the problem we came up with a protocol extension that would allow us to reuse the existing surface as a drag icon and allow a further reuse of that surface. Unfortunately I'm not the original author of this but you can read more details via the link below and this presentation can be found from the from this conference site. The implementation of the tag dragging feature is pretty complex. In the middle you can see a Wayland window drag controller that is responsible for dispatching drag events starting round loop and others. In a nutshell the workflow is the following. So start tag dragging, create Wayland data source and extended drag source. When the mouse is far enough from the tab strip create a new vl surface map it, notify the client about a new drag icon and then the client can swallow the request and start rendering the old surface as part of UI. If you are interested in more details prefer to the design doc and yeah it has quite extensive section that describes this behavior. Now comes the most interesting part. One may ask why do we need Wayland and here is why. So on the left side you can see Chromium running with Ozone Wayland on my Raspberry Pi 3 Model B Plus board and it runs on the Yocto Dunfield release with Western. On the right side you can see Chromium running with X11 on the same device but on Yocto with how it's called SATA. So it's a pure X11 session. There is a big difference between the two. In the case of Wayland Chromium could run at about 22 to 25 fps with 100 fishes. While Chromium on X11 could only run at about 6 to 7 fps while running the same demo. I can also say to you that at the moment I am working on the AAPI support for Ozone Wayland and once that is out there will be even more difference between X11 and Wayland and the current plan is to use existing the AAPI implementation that ChromoS with DRM uses. So you may ask how you can try Ozone Wayland on your devices and what's the status of the project. We have already implemented mostly all the features that Chromium on X11 has had and there are just a few missing and one of those is the tap dragging feature. We also have a build board that exercises test suits with Ozone Wayland X11 and headless backends. In total there are 10 or 12 test suits running with more coming later once we get more capacity for that board and this actually happens now as you listen to this presentation. And last but not least Ozone has been part of Chrom releases since version 87 so if you are using Chrom for developers you can try Ozone by using the runtime flags you see on this slide. Just run Chrom with dash dash enable features equals use as a platform and then dash dash Ozone platform equals Wayland or X11 if you wish to try Ozone X11 as well and that's it. So yeah we also have a meta browser recipe that we update on every release and you can try it from the OS system if you are interested in the October. So far so good but there is still one problem which is X11. So in order to ship Ozone for Linux and make it default it is required to also migrate X11 to Ozone. I don't think I'm going to go into much detail here but here are some interesting facts. So we started working on that in May or June 2019 and the effort included refactoring of virus X11 and Linux specific components so that they can be reused by Ozone. While doing so we were also enabling more and more features on Ozone Wayland like status icons and others. And in September 2012 we achieved a major milestone as I said Ozone became part of Chrom release for Linux. It is still disabled by default but it was something that users could start experimenting with. At the moment we are working towards refactoring components that are still X11 specific and sharing them with Ozone X11. Now most of the parts are shared by both X11 and Ozone X11 with X11 specific bits laying in a common directory under UI base X. The difference is that Ozone X11 accesses them through Ozone implementation while non-Ozone X11 accesses them directly. Once that is done and all the available test suits are enabled on the build bot we will start a finch trial. In the beginning about 10% of users will have Chrom with Ozone enabled by default. If the trial is successful we will increase the number of users using Ozone and once 100% of users use Ozone non-Ozone X11 will be finally disabled and removed. And I hope that the finch trial will start next year in Q1 or Q2 at latest. And by the end of 2021 Ozone will be default for Linux and non-Ozone X11 will be finally removed. Everything may happen earlier or a bit later but it was a lot of work for us. Just a single Ozone Welland counts for 400 patches with many others outside Ozone directory. Thank you for watching this presentation. I hope you enjoyed and if you have any questions please don't hesitate to contact me or my colleagues. Thank you.