 Thank you everyone for coming to our talk, we are going to talk about the work that Igalia does around embedded devices and yeah we are happy to be here in the embedded open-source summit where we also have a booth that you can visit later if you're interested. So first, just a small introduction about us, I'm Manuel Rego from Igalia. I have been working on the last decade on different web browser engines like WebKit and Chromium mostly also involved on server these days and here with me is Mario. Hi, I'm Mario. I've also been working for the past 10 years also in web engines in WebKit and Chromium mainly also for other years as well in GNOME related projects and Linux based operating systems so yeah web engines as well as my hats and I'm now overlooking the activities of our WebKit and WP team so hopefully it'll be helpful. Yeah, so first about Igalia we are an open-source consultancy that is highly specialized and we have people around the globe we have been found like more than 20 years ago so we have been doing open-source work for quite a while. We have a special structure, we are kind of a cooperative like company with a flat structure and yeah like spread around the globe I already said that. Anyway one important thing about Igalia is that we are the top contributors to Chromium, WebKit and Gecko outside of the of the actual companies maintaining the projects and we are very active in several different open-source projects like the different javascript engines to LLVM, Gstreamer, a bunch of them we will be going over all these during the talk and we are members of a bunch of different standard groups in many areas of the stack. Okay and about embedded devices these days if you think about them for a while you can see that many of them your web UIs like they are used for many different use cases but many of them if they have a display they probably are using some kind of web UI on them and the multimedia side there is just streamer project which is quite important in many of them which provides things like hardware acceleration and other features. If we go a bit to the graphics layer that is well and as a compositor very popular on embedded industry and then the RMR came as directly if you want to run to the driver and then of course having a good graphics driver for your project and your product is very important that has the good performance for your device. On the OS level Linux is like the main choice and the default choice and then there are even some new market emerging for embedded devices this virtual reality and augmented reality that is quite relevant also for us. So yeah like Igalia has an impact on many of these embedded devices through our work and we have been doing work on many of different things so here's a list because like I mean you can imagine now there are a lot of embedded devices like smart TVs, smart home appliances like cooking machines, audio systems or infotainment systems and cars, planes things like that, GPS and there are many many then this virtual reality headsets that are getting very popular and even like digital sign-up like kiosk mall things and the like. So yeah we are going to be going over the different things we have been talking initially explaining the work that Igalia does on each of them we will be passing so now it's maria. Yeah so yeah you're stuck with me now so um for the web rendering engines is one like we said before we are one of the top well the actual top contributors outside the maintainers of the main ones these days so let's start talking about WP WebKit it's one of them most it's probably the one that is mostly in embedded devices these days so what is this first so if you are not familiar quick introduction WebKit is an open source web and rendering engine for desktop and embedded is the web engine that is behind very popular browsers these days like on the Mac, iOS, Safari but also on other environments in Linux desktop and Linux devices it's I insist it's an engine because it's not a full browser this is something sorry that you embed to create a browser but not a browser itself this is making funny noises anyway so for WP what is that it's a port of WebKit for embedded devices with a focus on everything you need on those little machines right focus on flexibility being secure being performant or even you don't have a lot of resources because of that it's very important to get hardware based acceleration working and being able to integrate with specific requirements main use case of these devices using WP these days is multimedia because it's very common for instance in set of boxes but it's also useful in other use cases like server side rendering is a thing actually for different reasons like for testing for instance so the relationship between Igalian WebKit is a long one as well we started working to this project more than 15 years ago and we are a very big contributor in it actually we are the second committer to the project after Apple we are also the lead developers of two WebKit ports one is WP like I mentioned before we started that one from scratch in 2014 and the other one is WebKit GTK which is what powers browsers on GTK based environments I say using the GTK toolkit so besides that we also do work at a higher level on the implementation of web standards JavaScript features we also do a lot of work in particular on the G streamer based stack of WP to integrate with different requirements of media pipelines in the two ports we also implemented the accessibility support on Linux from scratch on WebKit and then we did a lot of things like support for 32 bit systems that is important for older devices performance improvement backfishing and so and now if we take everything together we've embedded as well Igalia WebKit embedded so what this means is that we work a lot of development and maintenance of RDK based set of boxes which already cover a lot of devices in the world we have been working on hardware accelerated SVG engine a new one for WebKit that put things into layers and makes everything more performant and GPU accelerated like I said we work on backends for this streamer but in particular to support DRM environments like MSE or EMA we implement of course custom backends for hardware for customers that they needed and recently we started actually also experimented with Android for WP running on Android and we do a lot of maintenance for downstream folks and before passing to Rego back just this is a demo of the new SVG engine for instance you see it's giving 60 frames per second what is interesting about this one is that it's putting different parts of the SVG into different layers and sending all those layers to the GPU so that the GPU can do the compositing and this is the reason why it manages to be much more performant that the current version of the SVG engine that doesn't doesn't do anything like that is all a single layer so this has a lot of potential for desktop machines but also for lower power devices as you can check it actually in our booth if you want later okay so continuing with the web rendering and the choices you have for embedded devices there is also Chromium and yeah of course you know Chromium but anyway it's an open source browser it's not just the engine it's the whole thing you have the engine but also the Chrome outside it and it's available for different platforms most of all the popular ones like Windows, Mac, Linux or iOS and it's the base for several other browsers I mean like Chromium is the main one but also for Edge or for Opera and Samsung browser I mean for some kind of applications like Chromium embedded framework or Electron it can be also used on embedded devices for particular use cases like for example the automatic great Linux project AGL use use it for the for the the UI and then about the Galleon Chromium this is again a long relationship because we actually were working on reading WebKit before the BlinkFork and all that so it was kind of related but yeah like if we just talk about Chromium 10 years of contribution with a lot of people on the community and getting integrated there we are the lead developers of the Wayland native Wayland support on Chromium and we have implemented different web platform features CSS Relayout, MathML, some of the big ones with many others and JavaScript features in the in Chromium and V8 then on the accessibility support side not only on Linux we have been improving and fixing issues with accessibility on Chromium in in all the different platforms and then yeah apart from that we have been doing things like around performance, code health, refactoring, generic bug fixing and things like that if we think about Igalia and embedded systems using Chromium we have ported it different hardware platforms for different customers we have deploying Chromium route web runtimes in things like smart TVs with web OS for example we are active members of the automotive Linux project where we are like maintaining the the Chromium layer there and we also maintain downstream forks like sometimes people need specific needs and they want to to keep things something like they cannot move upstream but we can help them to keep downstream because Chromium is a big project that moves very fast and doing like databases is a costly operation so here we have a small demo of this AGL project like this is a flutter application running on top of Chromium and basically this is like an infotainment system for the car basically like you have like information about the speed or things like that and yeah basically this is using Chromium and we are working on that and then yeah like thinking on web rendering engines Igalia is also involved on servo maybe you don't know about this one but anyway I will explain about it so servo aims to be an independent web rendering engine that can be modular and can be used for embedded devices it's focused on speed and security it has an API for embedders it's cross-platform right now Linux, Mac and Windows and one particular thing about servo that is quite relevant is that it's written in Rust so it makes the only web rendering engine written in Rust somehow like the other ones like the popular ones are all C++ so if that brings some nice features from the language that are very useful for for servo too that are like memory safety and concurrences so a lot of things happen in parallel in servo which is not that common in other browser engines maybe you already know a little bit about servo history but just in case this was started as a research and development effort by Mozilla back in 2012 there were Igalia was involved a little bit during a few years on that period in collaboration with Mozilla helping with different parts of the of the engine and then in 2020 like Mozilla stopped it to work on servo and they kind of transferred the project to Linus Foundation and at that point the mission for the project was still the same that initially nothing changed there but the project was getting kind of a basic maintenance mode for a couple of years there were no actual a lot of activity there or anything like that so this year Igalia took over the maintenance of servo starting in January and we have a roadmap for things we are working on like upgrading the main dependencies that haven't been upgraded for this long period of inactive like these are stylo with render and spider monkey also working on improving the CSS test parse rate so we can render more web pages on it and we have plans on the second part of the year to work on some experiments with embedded devices how to embed servo and some applications things like that and also start to work on adding adding Android support to the project and yeah about how this relates with embedded devices more directly like maybe some of them just need a small fast and secure web view and servo can be one of these because it's a way smaller project than chromium or webkit so if you just need something very simple and you have a control environment where you know which kind of widgets or web components you are going to render on the web on the on your application your web application maybe servo can be good enough for your needs and can be like something very simple to add and then it's also a good I would showcase for like advanced web features and also a nice place for innovation in the way of how do you implement these new features looking for more paralleling than other engines and things like that so things like web gl web gpu or or web xr can be run on servo and yeah they can you can try to make them like good there and can do them even faster than in others because it's a smaller project and then like if we're thinking potential use cases like if you have a kiosk mode application you just have a control environment to run there a particular application that should be fine or some ui frameworks for example in rust there are a few popular ones that that use a web view underneath it would be nice for them to also use a web view that is written in rust like servo it's not really for general browsing but it can be good for for other smaller cases so now they're changing the topic to multimedia graphics yeah I mean the if you know egalia for a while you know our history has been tight in the past years a lot quality to web engines and web browsers but that's not the only thing we've been doing so on the multimedia and graphic side of things we also been doing more and more work lately and getting more involved starting with the multimedia side this streamer is the main multimedia stack that we work with at egalia for customers so quick introduction again this streamer is a reference framework for Linux based multimedia applications not the only one it's a it's one that has provided is providing a flexible architecture it's a design based on pipelines and plugins so the overall idea is to get like a source of information at some point and then a sink destination without information in the middle you you can have different steps that process that information you can de max or I don't know process and call decode max again all that things so you put things in a pipeline in order to provide those operations you can rely on the core base of this streamer but you know but you you will use one of the many plugins that they have that provide things like specific support for specific codecs is an example in codecs yeah those kind of things so because of this it supports a lot of use cases uh the usual ones are media players and web browsers but even video editors for for yeah for doing something like on yeah video ads and streaming services all those kind of things um so our relationship with this streamer is also quite long uh more than 15 years again here we have a strong experience in particular with working with multimedia and embedded devices again which is where why we brought this up here actually in if you focus on just streaming web engines we are the top consultancy company there I guess unsurprisingly because of our working WP and chromium um but we are actually the second contributor in general in the past five years in terms of well patch is not the great metric that is one we have um then we are also the leaders of the development of um just in a way so I can see what key that provide different kinds of things not just video playback but also peer to peer uh possibilities with web RTC or you know adaptive adaptive streaming and these features that you expect of applications like Netflix like for media stores and media extension those kinds of things TRM based applications um then we also did work around hardware acceleration just streamer VA uh for internet and AMD platforms and Vulkan elements for Vulkan based APIs and last but not least we've been also investing for a while on just streaming editing services which is a component that allows you to implement video editors like PTB which we also contributed so the relationship of all this and and how it impacts embedded devices so um the new just streamer feature we've been developing upstream both in the in the core part and the plugins and those are crucial for crucial for for the embedding industry and then many devices like set up boxes or in flight in vehicle entertainment systems those kind of things um we do implementation of just streamer backends not just for working but for other web rendering engines this has a direct impact already millions of devices just if you just think of the million of set up boxes out there that are using rdk uh which is based on WP that's already a lot but there are even others not using rdk too um because of this uh it's important for us improving performance for hardware-solidated solutions because they again this is if you have a hardware which is low power which is most embedded hardware these days um it's very important to make the most of the resources there and and then finally uh other impact we have is that we do integration with other libraries so sometimes you you just get your particular media processing unit and you want to integrate it in a just streamer based package line so we also did that for hardware um and on the graphics side of things um about we work a lot of missa as well we have a graphics team working here uh full-time uh what is missa quick introduction again is an open source library an open source implementation of the open gl and vulcan apis uh which includes drivers that you can communicate with different graphics cards so graphics i'm not talking about kernel graphics i'm talking about higher level ones um so what is open gl and vulcan you might say open gl well both of them are close platform apis you can use to implement 3d applications and it exposes the hardware of the gpu in some ways you can interact with and it's developed by the crawlers group uh which iagalia i forgot to put it here but um it's also a part of that group so the relationship between iagalia missa uh not as long as in web engines but still fairly long like nine years more than nine years contributing to the project uh we've been working on the development and maintenance of missa drivers for for open gl and vulcan uh aim at you know a different graphics card different gpu size as i said before and we also been doing work around conformance test suites it's important that you you have a driver that is not just correct but also is complete so we always work on spending the api coverage whenever needed um and now how this impacts embedded devices again um well the driver's development that we've been doing for open gl and vulcan for different gpu's already had a huge impact in particular in in in some very common devices like the raspberry pi uh we work on the open gl and vulcan drivers there and in particular uh on the on the raspberry pi 4 we developed the vulcan driver from scratch started in about three years ago two years and a half and now is vulcan 1.2 conformant we also work on other platforms as well we also have demos there uh in our booth for the android phone of our vulcan and and open gl driver and drivers based on missa which is the free dino and the turnip drivers and for vivant as well so other things we do and in the missa side of things we we always want to enable like hardware acceleration on all the devices so at the graphics level we also have to work on optimizing some pipelines and again we have to do work integration work with underlying pipelines like waylang is often the case but sometimes we just go directly with drm and kms um and yeah and the last part last component i'm going to talk about before passing it to drago is about our work around operating systems this is a very generic thing so uh it's like the the final box where we put everything else so we in this case we are talking about mainly the linus kernel which you all know what it is but uh just highlighting that is the most common kernel using embedded devices and then other components like you know windows systems drivers sandboxing uh technologies test of integration so this this basically covers like a wide a very wide area so what is our experience here um we have more than 10 years working in the lower layers of the kernel but even more experience with distro related work like as devian developers and maintainers uh and actually we've been working maintaining packages in devian in like the common architecture for a long while we also are the maintainers of the rift 5 ports for devian which is still an official but it's on its way to become an official port of devian for to not relate in the next future short future um we are also the maintainers of the virtual kms component which is uh kms uh kind of kms that allows you to basically deal with the with machines that doesn't have a display attached which is again useful for server side staff or for qa you just want to emulate a display with a connector or a particular size of resolution you use vkms for that we also contribute to the linus kernel drivers for different gpus again this is not the misa drivers these are the ones at our lower level so we also work there and then we do a lot of other customizations in distros improvements in power management file systems and this many of these are in the context of our work on the steam deck um which we also have in the booth so we're welcome to come there and I can talk a bit more about that and then and our impact on about this and better hard to summarize uh we picked just a few so um in particular around build systems for instance too some framework we do work a lot with jocto and build root we also do uh work about linus device drivers not just for dpu i mention in those all the time but also other kind of cards as well we work on the creation of linus distributions as well and file system when needed um like a root file system not just you know distro related stuff it's not always about working on an already existing distribution and we do a lot of adaptation to specific requirements that again require like connecting different parts of the operating system um and we well we also port software to a specific embedded operating systems I guess I'm saying operating systems a lot that I'm sorry about that okay so yeah like this um this point is about the beautiful reality and augmented reality like we mentioned like you are also involved on this area that is a kind of emerging these days and we are basically uh we have this browser that is called walvick and basically walvick is the open source uh web browser for virtual reality like some of the virtual reality headsets have their own browser but if you look for an open source one you will probably find only only walvick there and it's not only a browser that you can approach the web but you can also it supports also immersive experience like web xr 360 videos and things like that again this was a project that started by mozilla in 2018 and he got it to cover maintenance of it and fork it I mean it was five four reality initially now we forked it and it's called walvick and we have been maintaining it since early uh 2022 and the good thing now is that if you have one of these um uh virtual reality headsets it's available in the app store so most of them so like the most popular one like the meta quest picor go away you can just go to the app store and install web walvick and give it a try and play with it and it's supporting I mean it doesn't support all the devices because you need like kind of adaptations to each of them but anyway it supports more and more devices as time passes so like if you have like the meta quest or Huawei glasses picor devices it will work there and you can give a try we also have a picor device on the booth so you can come pass by and give it a try to walvick and basically this is like how the walvick ui looks like it's kind of a different than a regular browser you are in an immersive environment and you can see like different three like three windows in parallel we are working to have even more flexibility here but right now you have like three windows you can have in parallel and you can interact with it and you can enter inside web xr games or things like that inside the inside the browser directly and okay yeah like now yeah to the last point like to the last part of the of the presentation like we have been talking about all the work that iagalia does in different areas of the stack related with embedded devices and yeah we are going to talk here about how we see a little bit the the future around these technologies so one clear thing is like open source is like totally attached to to embedded devices and embedded industry like it's used everywhere there are big projects like the linus kernel like chromium or work it like very big project that are uh baseline for the the rest of things you develop on a on a device and your applications and there are big organizations that are contributing upstream so that's very positive because you can innovate more do more work and there are many many actors collaborating together to improve to improve the solutions here then we see that yeah the web platform is is very relevant for embedded devices like web engines are getting more and more important because they are used in the embedded devices a lot not just as general web browsing which is also used sometimes but also for web based UIs like we we talked before like rich applications things like that that that are using now web UIs instead of of native widgets and things like that there is a very nice thing about the web that is open develop it it has been there for quite a while so it's quite well established a lot of people know about it it's always under continuous evolution so new things are coming and thanks to all the news from the embedded devices some new things related to this particular industry are also coming into the the web platform and there are some alternatives here like having WP server and other alternatives are bringing diversity to the web platform which is something very nice to have because like the ecosystem otherwise will become a bit uh monolithic somehow yeah and i mean that's one thing as we see like open source and and the web platform relevance we see that we also see that our vision in the future and then also distant future is that highly specialized expertise is always going to be a thing like you know embedded devices will keep having different sets of requirement different specialized needs will always will require people with specialized knowledge and companies to to be behind them and be able to work on that um especially with you will always have a need as well to develop just new new features not just you know doing integrated work but you need to want to provide different use cases support new hardware optimize things and actually about optimization performance optimization will be also key about all across all the levels of the stack not just the web platform or the graphics there basically you want to optimize everything because you want your embedded device to work as as well as possible and be as maintainable as possible and have the you know well longest life as possible i guess so with all this in mind and just to wrap up some final thoughts on how we see the situation how we see our role in in this complex puzzle that it is um so let's start saying um it took a while to prepare this live because it's a lot of things we try to convey here um so first of all we do see an explosion right now of the interconnected devices i don't think it's a surprise when you can basically connect your voice to a light bulb and a doorbell or whatever so and this doesn't seem is going to stop it seems like it's going to be uh growing more and more and more so there's an explosion and that explosion is probably going to explode a bit more um at the same time um we also see uh the existence of lower power and constrained devices uh within that's going to be a constant i mean obviously um constrained and lower power devices get become more powerful but uh for different reasons i mean from like for budgeting reasons cost reasons scaling reasons there are many reasons why those devices will always be as constrained as possible so uh this is the this is one of the key things so we don't think it's going to stop do not when we are not going to be using beefy machines in our in our wrist anytime soon we don't think so um and then um there is this other three parameter we wanted to bring up which is the environmental impact um not just for power efficiency or business reasons just for the kind of social responsibility uh i think the entire society is moving towards a reduced carbon footprint situation right uh and yeah and so and that combined with also willing to extend the life of of devices which uh it's not being great so far but i think there are actually starting to be some regulations pushing also in that direction so if you combine all these things together uh that kind of suggests that again constrained and lower power devices are going to be here for a while so yeah i has to finish this thing um we see there is this kind of tension right between product features hardware capabilities and and power management in in that we'll always want our devices to do more more and better uh at the same time uh that's always going to happen in in hardware that is always going to going to have like a lot of limitations for different reasons like i said before budget cost scaling and at the same time we always want well i guess it all goes combined uh as lower power consumption as possible if it's a portable device you want more autonomy if it's not a non portable device we still want not to consume a lot of energy if you can avoid it right so basically there is this thing about doing more with less so how this connects with entire talk uh we were talking about how uh well how we are driving innovation how we are basically contributing to the embedded space using open source so with all the experience we accumulated in the years and we started touching here and there and and becoming key parts of different parts of the stack uh yeah we think we are in a good position uh or we want to believe we are in a good position to keep contributing to different parts of the of the industry in the in the near future so we've been doing it uh i think with more or less access in the in the past few years um i think now we are going to keep doing that um yeah basically these are kind of our main let's say i don't know how to say pillars maybe so yeah yes and that's pretty much it so yeah thank you everyone thank you if you have any question we'll do a few now and then otherwise if you can find us on the booth is in front of the help that's just in the in the third floor so you will find us easily so yeah i don't know if people have yeah i don't know if you have any question or comment or curiosity or people remotely i don't know if they can ask questions somewhere or anything like that it's become a kind of a historical speech in the end yeah uh the question is if the if the wp and the working engine is using wallbeck itself so the answer is not at the moment uh wallbeck is uh fork of fire for reality when we took over maintenance from osilla so it it uses gecko and at the moment it still uses gecko but one of the main things we wanted to do is to decouple that and allow other web rendering engines so one of the changes we've been working on in the past few months since we took over is pre bumping the architecture to allow different back ends so the expectation is that uh we think the first back end probably we are going to try non gecko is going to be chromium in the next few months we should have something working uh and and having wacky in particular wacky with wp most likely it's also in the pipeline but i i wouldn't say this here don't call me on that but uh yeah the plan is to allow to do that yes the quest yeah the question is why would we would move to wacky or chromium if you already have gecko well i think that's a tricky question but i think i can answer that one thank you okay yeah like there are different reasons but one of them is like mozilla has stopped it to maintain the webex are stuck on on gecko so they since they stopped working on the project they are not doing anything there of course we can start to do patches there but they are like lagging behind features compared to chromium and also performance fixes and all that so we want to give a try to chromium compare both because we don't know if it's going to be better it looks like in the first test it's way faster but we don't know and also has some more features so we want to give it a try it's not like a make decision that we are going to switch and forget about gecko or anything like that but we're having like making this work to have like avail i mean like have an architecture that allows different backends and we can play with them and see maybe for some specific device the needs of one is are better than the other or things like that so we can maybe have flexibility in the future it is still to be seen because it depends on on different things that yeah any other comment or question so i'm not sure if i can summarize the question so okay now because he told us to repeat the question for the for the remote people so the the the question is it is about uh whether we are just focused on linear space devices or and or other yeah we are back but somehow this is not working there is if you're looking at things like all right right so no i mean do you want to answer you can repeat so yeah i mean the question is you the the question is whether we are just focused on low power embedded devices what stephen is saying is like there's not just just uh linux based devices but also devices with all their much further constrained hardware and not even linux like more basic than that and the question is whether igalia is also looking into that or just focus on linux right so right now i think we can say that we are quite focused on linux based operating systems for embedded devices maybe we haven't gotten uh that lower in the stack um so yeah that's i guess that's that's that's the easy answer really but nothing is discarded i mean years ago we we we were doing uh get on work only not web engines and now we're doing many other things i don't know but right now it does not a case more questions remote is there any question remote no that can be good or not okay well thanks you very much thank you for telling our talk thank you