 Having you mean it's when you are so age 5 minutes When you're missing 15 Okay Okay, so we're not gonna see it here, so I'm gonna go there. Yep the keys work. It's just doesn't play Oh Okay, so we have Mitri Marshall and Marcus sigla signal right They're gonna be talking about the portable graphics abstraction in rust So hello everyone, I'm Smith from also a graphics engineer at Mozilla I work in Firefox Rembrandt there and before that I worked in a game-deaf industry Which inspired me to start writing a graphics abstraction layer in breast And this is Marcus Who joined us recently to rewrite it and that's what we are gonna talk about The idea is to have a single abstraction layer To run on all the platforms that we have Unfortunately, the graphics API have been branched out. We have Microsoft Pushing the directorates 12 in the Windows ecosystem, and we have Apple pushing for metal Everything else is Adopting Vulkan quite successfully including the Android the most popular operating system And OpenGL is still alive as well And we want all of those to be available through a single interface in rust that is hopefully safe and fast It started in 2014 and The legend is we had three threads at the beginning and then we were fast-tracking through the evolution of the APIs As the native APIs did so first we got command buffers We got rid of the threads and then we followed with resource views pipeline states Eventually getting the display 11 and metal backends and then somewhere around 2017 when we started first talks about integrating into web render it appeared to us that Even though we already had the core layer split out of the Higher level rendering we still weren't zero cost enough. So we decided to go lower level We decided to implement the core to be completely zero cost and that's where Marcus comes in this is the main Roadmap of Our work and if you are taking one thing from our top, then that should be this slide. I believe At the center you see hardware abstraction layer. That's our new core. That's the name of the new core. It's Vulkan like rust API very low level quite unsafe That different graphics backends implement including Vulkan metal D312 and open GL These are the backends that we already have now and We are getting D3D11 soon enough as well The clients of the hardware abstraction layer are many aside from the user apps We have the Vulkan portability layer, which we are talking about a little bit later We have the warden test framework to ensure no regressions in the graphics tests and we have a web GPU prototype Based on servo that's using that a little bit more about the warden test framework. It consists of three parts First one is we are able to describe graphics resources and passes in readable Source which we use the rusty object notation for It says here's my images buffers Here's what I'm gonna do this with them in the following passes and then we have we can Load it all with a particular graphics back end so that it instantiates the resources on the GPU and creates the command buffers to implement passes and Then we have a ref test framework which has some very simple ref test defined and It runs the graphics passes on the specific back ends compares the expectations with the tests This is an example of the resources. We have an image With a specified format and dimensions. We have a rendered pass With one attachment and one sub pass Just as an example of roughly how it looks This is the contents of the pass. We have the setting the viewports the pipeline and drawing a single single triangle here Okay, so then why we want to actually use rust instead of for example C++ There are a few nice bits of rust and compared to the basic concurrency all const new features like pattern matching and their sensing which basically Tells you that every object in our API is sensing so you can access it from all threats and concurrently and There's also in in mutability, which we use to say for example, you can Change this object or you can't and it also encodes if you need to externally and force Single access because for example some objects can only be Access on one thread or at one time. So if you have if you see mutable access, you usually indicates that you need to External synchronize your object then Traits and associated types, which we use quite heavily which is each you will see later on and We split all the things into multiple repositories For example one repository is the hardware exception layer and then for each back end We have an additional repository and for it. Yeah, okay great So what we are currently missing is for example non exhaustive enums Which are basically required as we also provide an higher level abstraction for this Vulkan portability implementation and there are cases where Vulkan could return us Enum values, which we not really know at the current time of implementing which could come from a Future driver, which would return us this new Enum values then struct alignments and shaders which It happens every day that we misaligned some shader struct because and there's one page of Vulkan specification which basically says how you need to align your fields and it's usually You do something wrong there then const generics, which would give us the possibility to Use areas fixed-size areas more More often in the library interface. We basically expose a quite generic interface using into iterator, but Yeah, fixed-sized areas would help quite a lot for the user side to looks a bit better and Exclusive cargo futures at the moment. You could something could do something like I want DX12 and Vulkan to Run at the same time somehow, but it usually will mess up here and you usually only want DX12 or Vulkan So the back-end trite is our main interface. It's one large trite You have to have to implement if you want to implement a back-end we basically divided into dispatchable objects like the device or the command queue which You can call functions on and the basic resources like shaders Buffer image, which you usually just pass around and create and delete so for the roadmap 2018 we would like to become a Mozilla project because we want to integrate into web render and also into the web GPU part Which will Demetri will talk about and One side project is work on possibility. It's an initiative from Vulkan and from Kronos where they try to Do all the three three back-ends Vulkan DX12 and metal and Provide one RPE subsets from the Vulkan API, which can map to all these three back-ends and Our goal is to implement the Vulkan portability layer using our hardware extraction layer and we currently can already pass a few examples like triangles or pipeline example from Common samples from Vulkan samples and we are aiming at Implementing or hooking up the Vulkan conformance test Yeah, we can we try to hook up the Vulkan conformance test suit and Run run the tests against our hardware exception layer And that's how it looks from our side We basically have our hardware exception layer on the bottom, which is the DFRS side and on the top We have the portability layer and you can Interface it using driver In the implementation or using a static library and if you want to directly link it link it from a C library So a little bit more information about what WebGPU is It's a W3C working group which Microsoft, Apple Mozilla and Google are part of and as well as a lot of independent actors that is trying to design a new API To provide newer lower-level concepts onto the web for faster interactive applications on the web and There's been a lot of discussions over what the next gen API is gonna be and we I'm a part of the group and we don't seem to have a lot to agree on so far Everyone is sort of pushing in different direction So our position there is to have something either exactly like Vulkan or very close to Vulkan and Based on that we developed a prototype that is using servo, but adding in you web-idl based API That has basically Vulkan calls in it so you can go that low-level and create your pipeline barriers and descriptors on the web using JavaScript or WebAssembly and Run it in the browsers which in the browser which is currently a very specific fork of servo Our future plans with the WebGPU include trying to get a native application Trans compiling to the web using WebAssembly So a native application using Vulkan to be compiled using WebAssembly for the web on the browser So that they just trans in the browser so that you don't need to learn a new API that you can just use Vulkan on the web Now a little bit more about the ecosystem that we have and since we seems to have a lot more time left that we're gonna stop on Each of them closely There is a few engines in Rust at the moment. None of them is major enough But they're very promising namely good game easy amethyst and 3rs All of those use the older Jeff XRS at the moment where Amethyst is already transitioning towards the new hardware abstraction layer It's extremely promising to us is they not only Are porting to the hardware abstraction layer? They're bringing bringing the higher levels for us to use for example a Library that manages memory for you so that you don't have to care about Particular memory types that the driver exposes you just say oh, I want this buffer and the library handles the actual heap management and allocation of the memory from the driver for you or a frame graph sort of thing which allows you to describe all Your graphics workload as one big graph Which is then automatically deriving the dependencies the memory barriers for you and Possibly schedules it to run on multiple hardware cues for execution in parallel So this is what amethyst folks are bringing on the table, and I'm sure a good game easy and 3rs will follow and use hardware abstraction layer in 2018 other than that we have a very interesting plans with volcano as a Project that we are often being compared to in fact now the comparison doesn't make much sense because we are lower level We are implementing Vulkan and Volcano is based on Vulkan. So what we want to do is to Let users use Volcano on top of gfx hardware abstraction layer So in a sense, it's a drop-in replacement. The users will not need to do anything and they will instantly get access to Apple and Microsoft platforms potentially Xbox one as well So that's something that we are working on right now and finally web render is Mozilla research project that is currently being integrated into firefox it is in need of better control of the latency resources and More direct access on Windows platforms for for example playing a video efficiently so our Vulkan portability or graphic abstraction layer makes perfect sense to be a part of web render and We are already working on that. We have a group in Zegat University in Hungary That's 14 by Brenda and they have an exciting progress so far so they translate the shaders automatically and They already rust they already run our ref test framework. So we're looking forward to see web render running on how this year potentially upstreamed as well to Firefox and We have a very lively Gitter chat, which does not have a link here But we have also a blog post and you can just jump on the project page on github And it will have all the links that you've you will ever need We are always happy to answer your questions and right now I'm going to run a demo that Marcus have reading written the demo features quite a few compute shaders and some rendering as well and It's fairly complex it simulates ocean water and it runs Exactly the same on D3D12 metal and Vulkan using the same shaders and the same API calls So that is it Ready for questions So the question is why do we consider it a zero cost abstraction layer? It's zero cost in a sense that if you want to It to be backed by Vulkan then we don't do any Hip allocations and we don't do any smart pointers Management for you. It has explicit like allocation and free calls for resources Of course There is certain warts that we have to do in order for the same API to run on D3D12 and Metal so this is not exactly zero cost in that sense This is as low cost as we can afford but this is this goes in line not in our own reasoning but more in line of the Vulkan portability Technical subgroup and the design that we're doing as a part of the group. So It's very little minimum overhead. We can't afford any Extra GPU work for whatever we are doing Is your question that we are losing quite a bit of command buffers in metal? Okay, so the question is how do we deal with the performance loss from not being able to reuse command buffers in metal? that is very unfortunate and what Molten VK is doing as one of our competitors technically is Having a like a software list of commands build up independent of the hardware list. So I Don't think we have that implemented right now, but that's the plan right now Are there any plans for the graphical UI toolkit? We are extremely busy with making sure that the hardware obstruction layer like the lowest level of obstruction works and our next steps are more in terms of Getting the like frame graphs or the flibries or the memory management libraries solid before we can think about the UI however, I expect Conroad to be able to be ported on JFX hull eventually. It's a graphical UI toolkit for rust as well as Our friendly amethyst folks are likely to come up with something too. So we hope for your help So historically before that you write we required we required sorry the question was how do we handle shaders with regards to different platforms? Historically we Required the users to provide all the shaders for the back ends that they use like separate shaders for metal OpenGL and D311 after it the real right as we are going for the Vulkan portability goal we only accept SPRV and From SPRV we generate all the back end specific shaders We patched the logic like coordinate systems and descriptors IDs according to the specifics of the back end that we are Translating the shader into We're using the SPRV cross as like a helper library to do that and our far future far plans would be to Have a rust alternative to that as well since that's the biggest non-rust component that we use right now So right now you supply your shaders in SPRV. Of course the most convenient way to do that would be to write it in GLSL Say 4.1 and have them convert it using GLSL to SPRV That's what Warden test framework is doing at least you write your shaders in like GLSL and test them against SPRV so the question the question is how does webGPU is going to expose Vulkan given that webGL is way higher level so yes, it's a hard question and Vulkan is a hard sell for multiple reasons on the web Not only because it's low level, but all the consequences from it is like it's easy to build a user signature based on What you query about what device can do and on the web it's not safe And there's a whole lot of concerns that not only make not only Vulkan But in general going any low level on the graphic side To slightly start breaking the rules of the web. So we're trying to find our philosophy there and Our priority is getting like the maximum performance of the web