 So start with the obligatory marketing talk so We are from BMW car IT. So we're the software or doing basically embedded software for in-cast ECU so really let's say software running in your vehicles for BMW for the last 10 years We have a kind of small office of 250 people in all Where we do software for a bunch of different ECUs But mostly focused around infotainment body domain and autonomous driving I'm Brendan Lothal. I'm part of the architecture team in in all where we specialize in The Linux platform development for our ECUs So my responsibility is the in-house Linux-based software development platform and Our general let's say aim is to develop reusable software For basically in-house ECUs to have basically reusable software across ECU generations rather than essentially starting every time from scratch So let's generate. Let's say our picture I think quick recap probably is really obvious for people in your automotive industry But we have a lot of ECUs with software in a current vehicle So there's actually a real picture from a new seven series showing you kind of all the large ECUs and the general placement So what we're really targeting for for some of these ECUs now is our general Linux distribution that we call internally node zero There's actually a pretty I would say standard big Linux embedded distribution. So it's it's your actual Linux base. It's system glibc It's nothing extraordinary I think one of the big things we're doing essentially and we want to have integrity protection for basically absolutely every executable we run and obviously using Hardware mechanisms for security hardware key stores this kind of thing And one of our main goals is we're trying to avoid Delegating early tasks to microcontrollers or two things that would then require a handle with the Linux So we're really trying to essentially run everything within one OS rather than essentially handing over certain functionalities or let's say Having multiple systems to maintain. So we're trying to say keep things simple And the last point is a bit inflammatory. So it's we don't actually really boot that fast It's just that we provide the ability to run early tasks and to actually run certain parts of our stack very early So it's not like everything is running really fast. I think we're not doing anything magical that everyone else is not being able to do But we basically just have the assistant design so we can run and order tasks in a kind of interesting way and lastly There's really kind of coming from the product side So actually we're really shipping basically this platform now in for ECUs in 2022 And I just want to kind of give you an impression of the amount of ECUs that we're really targeting basically now and in the future With our with our Linux platform. So you can really see it's kind of say growing pretty fast And lastly, it's it's really let's say we're focusing our design on let's say autosar or adaptive autosar free So we don't have any of this in our ECUs So a little bit more of automotive stuff So I mean I think everyone maybe knows Some people might know why we need to boot fast, but I think there's a lot of generic reasons Typically the answer is that the NHS TA has a requirement the review camera must be displayed within two seconds But I'm clearly that you when the two seconds starts when it stops But um and different manufacturers do it differently. I had from one manufacturer. They take five but I'm not here to judge Essentially There's actually lots extra reasons why we want to boot fast I mean, I think this review camera is just a typical use case and it has an eagle behind it But actually I usually take the example of them if I'm on a motorbike and I get on the motorbike And I press the start button or I turn my key. I want the system to come up pretty fast And I think this is a pretty more reasonable thing to imagine Or something like when you open the door and you have some projector showing you some lights or something The more we're putting Linux into basically more complex systems or systems Which are even not let's say relegate it to infotainment. You need to basically have let's say a quick A quick boot up time for some of the some of the infrastructure, which was typically done by by real-time systems So I think just to give a quick idea of what we currently boot really fast in our systems And what will the kind of use cases are to give people kind of an idea here's a kind of current high-end cockpit of a I think it's my 20 But basically things like the inference cluster the driver camera system The only video use case ignore the fact of showing a Navi And a head up this is typically what what we basically need to show the customer relatively fast when he's getting into his vehicle And I mean a customer with a vehicle is not like a smartphone customer. He's expecting this to kind of work relatively fast So yeah, I think it's a bit different to other use cases Now I wanted to qualify a little bit what we meant by modern security Initially with decent security, but I was told that's not very good. So it's now modern But generally what we want to try and achieve is integrity protection for all executables This is really basically Not just for I would say pure Hackability of our system, but actually it's really practical to know that the software that we signed is what we're shipping to our customers They should be super important and then no one's tampered with it on the way there We also want to basically use modern secure key storage mechanism. So this is Typically to protect our back-end access for Authentification of the ECU and especially preventing physical theft in one thing Which is quite important to us is that we don't want basically our cars being taken apart by someone and then sold in the black market It's also raising people's insurance costs. So Basically, how do we basically lock down our ECUs to the vehicle? And obviously we have more than use cases. So we want some DRM keys for providing some nice content to customers On top of this, we also want that's a mandatory access control. So we're using SCMX on our ECUs We also want to have IPC security policies. So essentially we want IPC to be Yeah, essentially have access control and security policies around it We're using Ethernet a lot in our vehicle. So more and more you'll see 100 meg gigabit Ethernet in our vehicles and we want to be able to somehow pair The ECUs again so that the communication between them is secure We're using basically IPsec currently for this Encrypting customer data. I think this is pretty obvious And the general let's say things that you should do in an embedded system So not running binary the route having minimal privileges for for binaries Which would require certain things? Um, I think it's let's say pretty obvious these days that with things like UN ECR 155 You need to be a bit more serious with this stuff and we need to document how we're doing this And I think this is that's a quickly becoming pretty custom and then standard practices in the industry So let's get booting So I think the first thing It's just a void doing it basically I mean cheating is way way easier than booting really fast. It's really really hard to get into a car in under three seconds I mean the motorbike example I gave is it's kind of a let's say a bit of a hack around this It's really hard to get a car in three seconds Also, what we're doing a lot is using Suspender RAM So str is kind of interesting. It's not necessarily as fast as you would initially think so It's actually quite surprising when you open a laptop lid. It's not even that fast And it's kind of similar on on ECUs It's obviously pretty fast and let's say it's taking I would say around one second on average to get basically everything back up in a kind of working order on our systems The problem being that let's say It's still essentially relatively slow and you still need to optimize Your your hard boot to cold boot times anyway because anyway your requirement is the boot in cold boot So it doesn't really matter if you can only do your requirement in str. You also need to do it in cold boot You also basically can't always str to basically protect the battery. So even electric vehicles to a problem You can't basically always rely on the battery as their customer will go holiday for like four weeks Or keep this car in a garage for like two months And it's yeah, it still needs to start relatively fast So it's kind of giving you double work The big advantage of str though is that you can have your whole application stack started really fast So the whole thing is looking really good not just your early use case And I think that's really highlighted by the fact that I'm giving you a real number here So it's really in in a current ECU. We have some about the situation We're basically only let's say getting our real early KPI down by about 500 millisecond by using suspender RAM And that's because you need to wait for basically the network to be back up the camera to be back up all of this And this actually takes quite a bit of time. So str isn't necessarily saving everything Also a lot of our message even does just don't really support str or do it really badly So this pretty painful And lastly and one way to avoid is hibernation. I think it's it's pretty interesting but generally Flashes are still not that fast And it's pretty difficult to guarantee a lifetime of over 15 years When you basically have this kind of ride cycles, which would basically allow you to do hibernation So at least we're not really managing to do this yet. Um, yeah, could be interesting for you in your use case So you really want to so So generally we're the high level issues, so at least the ones I kind of wanted to go through with you guys today A few kind of highlights that we found that I think are kind of generic enough to share to share with everyone and maybe maybe help people One of the first things I added is basically how we do our AB updates and partitioning of our system Partly because I thought I read a talk about optimizing bootflows and was talking about how AB updates were making you slower And I was kind of confused. So I thought I would add something Secondly, basically our integrity checks a little bit about how we basically load the kernel Little bit about how we do we do in it in Udev and then some let's say generic security things and it's not entirely in order So I structured it let's say and it kind of before Linux boots after the next boot in user space But let's have a quick look at kind of let's say how you get a rough timeline of your your high level things So I think this is basically a relatively achievable boot sequence It's the numbers a little bit wrong, but it's it's generally basically what we kind of target in a let's say modern system It's generally what we also see So we're seeing basically a first user space achievable in about one second with basically all security enabled Including the power stabilization phase. So this is really basically when the power goes into the ECU so the famous 12 volt rail which is not 12 volts and Yeah, typically we have a first stage and second stage bootloader and at some point the kernel is booted After this and we end up in roughly the second range And then we have to start orchestrating and starting the other things The first thing which is kind of yeah relatively interesting is secure boot these days is not really that slow. So We even have some vendors telling us that it's not even slower than not doing it Not really too sure. Maybe they don't want to support not doing it It's typically mandated any way if you want to use trust sign or if you want to use some of the hardware security features So you're kind of forced to do it anyways One of the other interesting things is that I'm actually changing your key size to basically a smaller key Typically doesn't net you quite as much performance as you might expect These hardware accelerators are kind of fast We had one use case where we benchmark So I thought it was interesting to give some kind of real numbers We benchmark basically move from 2k keys to 4k keys and this added about 40 millisecond So it's that say you have to decide whether this is good for your use case or not But it's it's generally pretty cheap It's also relatively cheap to do a secure boot on your extra microcontrollers around your platform, which is kind of interesting as an aside I personally thought that would be kind of a no-no, but seems it's working really well these days So we're also able to do secure boot on let's say other peripherals around us, which is which is quite nice I think the next part is that our modern managed flashes So EMMC's and UFS is are actually really really fast nowadays So I think we're talking about it earlier and something around 10 times faster than what we had to say a few years ago I'll let this really true, but it's it's for sure super fast. So these EMMC's in HS 400 mode and these UFS 2.1 and above are really really fast and What's kind of interesting is how they kind of generally mapped these days So this is more it's a more complex UFS type mapping, but generally The way that typically organized in a BSP is you have it's a certain section of flash Which is essentially sectioned off and typically marked as this enhanced memory type 2 is how the jet expect calls it But generally this just means that the let's say flash vendor has different segments of his flash Which he considers different quality and depending on how your vendor is doing this This could be quite good or it could be not that great But typically your the BSP vendor will put basically all of his bootloaders in this enhanced memory type 2 region to try And be a little bit faster. So typically it's using SLC cache to basically that segregated off into this this region However in quite a lot of cases now, it's basically the same multi layered stuff, but in pseudo SLC mode But it's still a tiny bit faster and maybe a little bit more reliable to put your bootloaders in there Some vendors are even putting the kernels in there and everything which is kind of interesting And yeah, this section is typically about a hundred under a hundred leg for both And typically we have two because we basically have an A and B side with some kind of partitions inside there And obviously then you have let's say past this section Rest of the lungs which is typically then segregated into let's say either your level two bootloaders or your kernel or just your root file system Let's say you are the fast But the really interesting thing is that the kernel and how it deals with basically how the label of these these modern flashes So the kernel would typically create your block device for every single segment of this flash So this means that even with one UFS and this is also the case in Android phones You'll actually end up with something like seven block devices, which look like seven independent partitions or partition tables and this presents a kind of interesting problem because Udev is we'll see a little bit is kind of slow and this gives you lots of events already and Also because these block devices are not necessarily enumerated in the same way at every boot This can give you some interesting randomness Especially if you change UFS vendor or if you do something else So one of the things that we're trying to do is essentially trying to minimize the randomness of this by Pre-allocating block devices to certain devices that we expect And we also essentially limit The amount of block devices that we are basically allowed to pass by the kernel Kind of very simple patches, but it's a nice way to essentially get you to be Not so random and typically in our ECUs we don't get hardware that we didn't expect the first time round So this is let's say super ungeneric and not a very nice solution But it's really practical to try and basically limit the problems you can have I think we only have really one use case where we have let's say devices where people can plug in I mean, we have a USB port so clearly you can get an extra device in But this is typically done way way later and doesn't affect the early boot flow It's it's actually really a good gain. I mean we gained something like 300 millisecond by basically pre-allocating our early block devices But it is true that let's say depending on your BSP vendor and how much they're abusing this kind of layout You can't have really a lot of partitions with the default BSPs There is one vendor out there that will give you nearly a hundred partitions in the default BSP So this a little bit crazy, but this seems to be how people are structuring things nowadays So I promised it I mean basically ab partitioning how we're roughly doing it I mean it's a little bit special depending on your sock vendor and I think some sock vendors are really much simpler than others I think ti had an interesting presentation showing how flexible they are. I would argue that it's just really complicated but Generally the way that these modern flashes make it really simple if you don't have a north spy for early boot which doesn't really gain you all that much these days is You basically have a register that tells you which working loan to go for so what you can basically do in your in your wrong Is just check what this register is as soon as you know, which basically boot on to go for then you start Basically this bootloader and this becomes your boot partition The really nice thing is that this bootrom is baked into the sse and nearly all modern sse support this type of AB layout Because it's essentially exactly what Android does Or at least what Android does a few let's say In certain vendors, but it's typically really let's say really supported by these bootroms So they basically check this register and then choose what they what they will do based on Exactly this this information and they don't go through some persistency on the north spy or something like this So it's essentially really nice because it can't really be corrupted and especially in development. It's really nice because you just can't screw this up Obviously the downside is that all your recovery and let's say mechanisms What would you do when you didn't manage to boot this first long things like this? You're basically locked into what the rom code allows you to do so obviously you can't update this so Your recovery strategy from a completely failed boot is a little bit. Let's say locked down to your vendor And obviously it's not really secure on its own, right? Um, you still need to say figure out how you're going to do your Your rollback protection things like this doesn't really not a solution on its own But it's it's kind of a nice way to do a be partitioning in a really simple way So then we get to Maybe some of the funnier parts. So one of our main strategies for the kernel is It's actually modularized everything we can so I mean, there's a pretty basic if you want to go faster load less So we try and basically get our kernel down to as small as possible And then have basically modules loaded for all the functionality that we absolutely need really early and then try Load the kernel modules for things that we don't need to early later. I think it's pretty basic We also have ECUs with a relatively large amount of RAM now So we actually see it's beneficial to actually hot plug RAM So to start with less and then add the the rest later Which I always think is quite funny But the kernel is really good at this. It's able to do this really nicely One of the things which I think is the most interesting is actually the scheduler during the boot up sequence This actually makes quite a large difference But it's really difficult to get a recommendation. That's basically applicable to everyone's ECUs or everyone's system But I think the nice thing with these embedded systems that you always boot in the same way because it's always the same system And you only need to test your system not someone else's so it's relatively simple to just play with it a little bit Have a look at what you can get Especially if you have a architecture, which is essentially a big little or even worse a fat big little Where you essentially have multiple cores of different let's say performance and the two big schedulers that at least we benchmark as it's more impelled and generally the The big let's say difference philosophically is whether you want to essentially Prioritize core migrations to your large cores and execute there or whether you basically want to keep Your load basically spread across your cores and not basically migrate around I think that's generally the high-level difference So one is basically giving you more core migrations and higher scheduling may ncs Whilst the other is essentially Giving you less of this but on the flip side has slightly less aggressive frequency ramp up So it's Some tasks may be a little bit slower. So you really have to kind of have a look at what what you can gain I can at least sell you for one use case that we basically ended up choosing Pelt and this gave us something around 20 millisecond gain on the display driver starting And we gained overall on that say the full system start up around 200 millisecond But this basically the full system started for the for the real platform, right? So then when you consider let's say everything the containers are starting afterwards and really your full system ready thing We gain actually nearly half a second So this is kind of interesting for essentially something that's pretty much free The other thing is basically if you prioritizing latencies for for some processes Yeah, you really have to go have a look in your use case and what your what your micro architecture is like But this can gain you some some interesting things So we talked a little bit about how we basically want to Do integrity checking for our whole system? I think that's one of the real big I would say new things that we really introduced with this kind of generations of eric use Where we're essentially copying exactly what Android is doing but putting that on the next so one of the big things we wanted to do is to use essentially integrity protection for all the binaries that we executing so all of our Redry partitions are basically no execs. So we have no binaries on there And generally we ended up with the enverity Which is working really quite nicely and we're using it essentially without an in the RD Which is exactly how Android is doing it in order to do this you basically need some kind of bootloader Which is understanding a little bit your your your structure? The nice thing is is that the ABB to format from Google? is available and most bootloaders are basically supporting this So again, we're kind of using the fact that Android is so pervasive in this market that this technology is available relatively easily and We found that in practice. It's it's really much faster than IMA and we don't require the use of an in the RD Which I don't have a good number for but we think is is a good way to save a bit of time De-enverity also provides us some interesting error correction capabilities Well, you have some forward error error correction inside the VB meta partition And so we're able to basically yeah Correct possibly a few bit flips if we had them so it's actually giving you some extra reliability And obviously you don't need to do any FS checks or whatever because you never touch the partition It's kind of nice It also allows you to skip kernel module signing So if you use the enverity and you use load pin correctly You are essentially able to say that well all my kernel modules are on a partition which has de-enverity load pin essentially blocks the the kernel from Using modules from any partition outside of the first one is loaded. So typically your root var system And it's also guaranteeing all firmwares loaded by the kernel will also be from that partition So it's really nice because actually then you don't need to do any extra signature to sign checks After this and of course because de-enverity is doing let's say on-demand Checks, it's not like checking the entire partition and then loading it You're not really slowed down if you have any kernel modules, which basically you're not going to load until forever You don't have to do the checks. Let's say in advance So this is really nice obviously with the restriction that you can't load things from outside your your first partition This is a pretty easy restriction. I think to deal with On to more fun security things So trust zone has many issues. I'm not sure I put the issue, but Generally TAs are like super Dependent on your BSP and then how it's kind of structured. Here. We're talking about one vendor in particular I'll let you guess the the big problem basically with accessing trust zone is that you can only do this through A user space application to actually communicate with the the trusted world with the secure world as they call it So it's actually not really possible To during your Buddha process of your kernel You need to start a bunch of stuff until you start the arbitration demon and then you can communicate with the secure world And the problem with this is that the arbitration demons typically are written and expect lots of things to already be started And if you start relying on them for doing let's say very early things like you want to start your persistency That's read write that's using a TA to get the key or something Then you're basically kind of blocked until trust owners really started before you can do things and this is typically pretty slow and Let's say that the more you're basically using via trust zone in your in your early the more you need this arbitration demon to be Up really fast and typically Trust zone itself or the TA is a pretty fast load and they're typically relatively well parallelized Even though they do steal your CPU cycles during the boot up, but just typically relatively cheap still the problem is really this arbitration Demon that's starting and it's really difficult to basically get this Yeah loaded in a fast way, which is really really transparent with the rest of the user space I think one thing that we're really looking forward to and it'd be really interesting to see if We managed to let's say get into this kind of world. It's things like arm system ready which is basically kind of let's say Specifying how basically an arm sock should kind of boot and how the boot flow should kind of work And this would really help us standardize basically what what you're allowed to do in this year one and the other two modes And where the let's say secure world really starts. So I think this is One of the things that would be really interested in the future to see if we can Get some good Some good improvements there, and I think just to give you an idea Typically, I mean on this kind of product. We're talking about yeah nearly 150 millisecond just to start the Trust on let's say Application images and the hypervisor to actually run its trust on applications inside and then the trust zone arbitration Demon is starting even later So it's basically really really slow for us to use this now some of the maybe more more funny parts that we do so Let's say once we're let's say outside of the kernel and in well into user space We're using system D and system D is is really really nice and really pretty flexible. Let's just do some some really interesting things But one of the things basically we try and do is start things in what we call the early target and essentially What we define as an early target which allows us to run things before you dev enumeration has really started and before Essentially, the system in the target is up and that allows basically us to reach boot times that Pretty good because we can basically start some user space stuff before basically system D would typically allow you to run things Obviously Every application that you start in this in this early target of ours Needs to have really minimal dependencies and you'd be really careful about what you add in there You can make crazy things And just to give you a kind of example of what we kind of start in this early target we're talking about things like deboss some system D network D services Our runtime variants so typically our ECUs are using a kind of concept of one image for multiple variants And essentially one of the first things we want to find out is what ECU is this Or let's say you have some basic information of what this exact hardware variant is and what we should do about it Communication to basically your IO controller, so you probably have some white controller that does some low-level stuff Where we need to kind of communicate with We're using some IP to communicate to our other ECU So we need to start this pretty early to kind of do most things and then some general network management and it's a startup Devices which are a little bit custom to us But generally just give you a kind of idea of what you need to start and then let's say the system for us Is I would say partly functional for some applications And after this we're basically then resuming the normal kind of startup of system D and Allowing it to basically follow. It's really nice kind of ordering And this is really kind of flexible. We have some kind of special boot modes We're basically our kind of general targets, which are basically application flashing and basically remote software update targets Which we basically use to organize our startup Let's say as you put basically in a normal system D system nothing nothing too crazy So we kind of alluded to it before but you dev is is really noisy And actually slows us down a fair amount in this in this early startup phase We obviously really like to use you know, it's not like we try to avoid this on real purpose. That's why we still use it but the events are replayed in an order, which is not fully predictable and The filtering of certain subsystems or the prioritizing of it Basically is really tricky It's also Yeah, really difficult to kind of stop you dev triggering events and then having them retrick of them or retringering events that you've already seen To basically avoid too much noise It's kind of really difficult to them to get this working and We have a kind of number of poor requests that we we try to push towards system these are trying Yeah, make it a little bit more configurable basically which events we wanted to see and how we could re-trigger certain events And I think the problem is always that it's not very generic and I mean, it's totally true it's our Our patches typically trying essentially Yeah, configuring for a certain target where you really know what's going to happen and what events you're expecting And it's really difficult to essentially have a Super generic way of configuring things. So then currently we're on my v2 of the patch sets Which yeah, I think it's still not going in So it's yeah, I think it's really difficult to kind of get a good compromise between something Which is really configurable and generic for any kind of system and something which is that say Yeah, really did say Fast I think there's been some improvements in In filtering for subsystems, but it's still quite a bit slower than what we currently have So we're losing something like 100 200 millisecond basically on on you dev in this really early stage Which is kind of yeah other stage we really want to gain basically hundreds of milliseconds because it's quite a lot of time The next big offender that we have in our user space Is the pulkit is really really big It's using JavaScript for its configuration And it has loads of cool features and you can do loads of crazy stuff with it But it's like over 50 megs and it's Yeah, pretty difficult to avoid it I mean obviously we're using deboss because we're using system D and for other things so we want to have basically some way of Having secure IPC around it and then say making sure that not everyone can call everything on on deboss I mean it's clearly a bad idea We're really basically struggle with with pulkit so we we tried to use these recent patches to replace most JS with duct tape Which is actually really nice. I mean this is replacing most js thing is like 40-something megs currently 47 And duct tape is something like 300 kilobytes. I mean it's pretty big gain already But you're still passing basically a JavaScript configuration file and this is pretty still slow And especially on smaller issues We basically were really struggling and we're losing nearly one second basically in pulkit startup And this typically then blocks us for everything which is using system D Because if you have some kind of let's say small small thing that needs to basically figure out if services started Or you want to basically do anything on deboss and system D that you don't run as route You're basically waiting for pulkit So we really kind of struggled to see how we can make this work and in the end we came up with making our own so It's not quite ready, but we will basically be opening some open sourcing this later Basically, we wrote a small pulkit replacement demon called small kit It's pretty much inspired by the old osprey project group check. So essentially it's a Yeah, kind of one-to-one replacement of pulkit, but with a lot less features and supporting a lot less configuration Yeah, it's a bunch of missing stuff And our plan is to basically provide this as a virtual provider for pulkit in Yachto So it's basically a complete drop-in replacement for pulkit And at least yeah in our experience basically if you want to do anything with system D this nets you basically nearly one second for Yeah, before you can basically use deboss. So it's it's really really interesting And lastly Yeah, one of the things we're doing a lot is Containers I mean I think like everyone in this conference is doing we were also trying to let's say containerize our world and applications and I think one thing I wanted to kind of highlight is We We wanted to basically have a look at let's say which container framework we should be using so we were using LXC but we thought maybe we should be a little bit more modern and and try and some nice new things that people are using and search we We had some benchmarks done to try and figure out let's say Yeah, what could be basically another modern framework that we could use those to basically get us some nice features and kind of overlays It's kind of stuff and and how it worked basically on our embedded ECU's And the results are kind of pretty clear We really can't use anything, but what we currently had because everything else was really really really slow and Yeah, one of the things we kind of let's say see is that a lot of these container frameworks like this are really not let's say Optimizing for the kind of boot times that we want. It's good. That's a very different applications. They have in mind and It's essentially a pretty difficult for for us to move away from essentially Very basic name spacing setups With basically a pretty low level demon, which is let's say pretty small and all of this overlay affairs and kind of fancy Orchestration stuff is just yeah way way too slow on on small hardware So that's a bit of a sad thing I think we kind of let's say we'd like to see if we could kind of think of a container framework Which is a little bit let's say lighter, but a little bit more fairly featured than basically what we're currently doing But currently I would say we're really not there So I think it's basically all I really want to talk about today and all we really had I think one of the things We're really interested in is basically who else has this problem So we often talk about let's say Our boot time is a kind of problem in the automotive industry and we're kind of curious if anyone else has these problems I mean It would be kind of interesting to see basically what kind of solutions We can maybe try and co-develop together or see basically Yeah, how we could basically for example get some more patches into system D up stream see how we could Replace let's say pull kit for our use cases for certain things We'd be really interesting to kind of find out from from you guys Let's say what you are doing and if you have let's say any ideas on things that we could collaborate on to basically try And make let's say generic user space for pretty standard Linux a little bit faster So, yeah, please feel free to like come and talk to me. We also have our booth downstairs We're showing basically the most of the ECU's in a kind of infotainment cluster of the new seven series So you can try it out a little bit yourself Yeah, and if you have any ideas or things you think we could we could collaborate on would be we'd be super interested Yeah, thanks a lot for listening Any questions or? Yeah So I think the questions around the Yeah, Debian Polkit, which is not based on the JavaScript MOSJS stuff So actually we evaluated that as well and actually it is a bit better. It's not amazing, but it is is better The problem we see is that it's like kind of not really well maintained And we kind of let's say not really too sure whether we should be using this kind of partly dead upstream So a little bit. Yeah, and it was one of the options basically But we kind of decided that there wasn't really enough of an upstream to try and use this So, I mean if you look at like how much code you need to basically do the really basic stuff that we need for For a Polkit replacement because we don't need like all of the stuff like where you ask the user basically like whether he wants to Authorize certain apps things like this, right, which is much more complicated We really just want a static config loaded once and then don't touch it again So you can actually do this with relative to small amount of code So you can be a bit faster, but yeah, it's another thing you can do this. That's a pretty cheap and easy Yeah, I mean you can also use group check It's just it's also very much dead for the last like two and a half years, but it's actually fully working Am I allowed a question during the stop or is that not allowed? Is Yes, the question is basically about putting certain applications basically in another OS with a hypervisor and then Loading basically your your fat OS later, and I think this that's a typically something you can always do, right? I mean, it's basically what we try and avoid because it's then making I think two big problems one You have to maintain two systems. You also need a hypervisor Typically is not entirely free both performance and cost wise But also then you you have this really awkward thing where you have to do a handover Because the typical review cameras that we have when you have so you have to kind of review cameras, right? You have the one that's really early for basically getting the two seconds Which is typically the pretty basic review camera that I hope most people don't see And then you have this kind of like enhanced nice review camera with all the kind of bells and whistles And if you show this really early one and then you want to basically show kind of upgrade the customer to the nice one after the boot You kind of need this kind of complex handover and typically you end up if you have to switch for OS's and you're switching everything It's typically difficult to make this look really nice and so Optimizing be it spending a lot of let's say time and money optimizing this basically plus the extra OS plus the hypervisor It's typically difficult to argue for when you can essentially just try and make Linux as fast as possible Just to get this problem out of the way. So specifically for the RBC. I think it's Yeah, you can of course do it this way. I think a lot of car makers are doing it this way But we think with the kind of boot time we can achieve it's kind of we can avoid it basically and that's I think a better solution but Yeah, if you have a hypervisor anywhere and you're doing things can work Thanks, then thanks a lot