 Next up we have Emily Ruppel. She is a research scientist at Bosch Research. She works at the intersection of embedded programming and embedded languages and programming and, oh excuse me, embedded systems and programming languages. And she's going to talk to us today about WebAssembly in real-time safety critical systems. Please welcome Emily. As was mentioned, my name is Emily Ruppel and I'm a research scientist at Bosch Research and today I'm here to describe what happens when WebAssembly meets cyber physical systems. So the very first thing to know about Bosch is that Bosch builds things. Depending on who you are, I tend to describe Bosch as that company that does drills and dishwashers or all the parts in a car but not the full car itself. And what a lot of these devices have in common is that they can be categorized as cyber physical systems. Fundamentally they've got a microcontroller or microprocessor that's talking to a series of actuators. These are things like lights and motors and pulling in information from sensors. These are things like cameras, accelerometers. And fundamentally what's happening is we're using computation to interpret and influence the physical world. The thing to know about these systems is that there tends to be a very tight real-time deadline on this sense compute process circle because we're interacting in the real world. So we need to keep this within tight time bounds so that our systems operate safely and the way we expect. The next thing to know about these systems is that oftentimes they operate not standalone but networked together. So if you take a modern vehicle what you'll find is at minimum, bare minimum, dozens of electronic control units or ECUs working together to make your car drive forward correctly. The trouble with this kind of system is that it's highly heterogeneous. Within your vehicle you will find a full range of computing power going all the way from your vehicle computer which is approaching what you'll find in a laptop all the way down to low-end ECUs. These are tiny microcontrollers running say your windshield wipers. And the trouble with that is that the software ends up highly coupled to that underlying target hardware. I can't move it around at all. And this is a problem as we go to add additional layers of hardware to this system. Already we're seeing proposals for roadside or parking garage resident servers that could allow us to offload autonomous vehicle functionality so you don't have to have that expensive hardware in your car. Perhaps even sooner and a little bit more realistically we're seeing fleet management who that's relying on data processing happening in the cloud to get a full sense of what's going on within the vehicles in a fleet. And this is exciting because it expands what it means to write automotive software but complexity just explodes because these software stacks were never meant to handle computers ranging from ECEs to the cloud ECUs excuse me to the cloud in a single application. So that's the first question we have here is what software solutions can we use to simplify networked safety critical applications? So if you have a complicated hardware stack that you're trying to hide, virtualization is one of the first tools in the toolbox. So that's what we look to. How can we virtualize all of these underlying hardware pieces? The trouble though is that not all virtualization solutions are going to fit across this broad spectrum. For instance, hardware level virtualization like virtual machines really only good for the server side. Operating system level virtualization like Docker will fit on our most powerful ECUs but isn't going to work on the smaller ones. So that is where WebAssembly comes in. Using application level virtualization we can finally virtualize apps and run them all the way across this spectrum. So at Bosch research that was the question we set out to answer is how can we use WebAssembly as our form of ultra lightweight virtualization to cover all of our hardware platforms? So throughout the rest of this talk I'm going to describe how we are using, we are interested in using WebAssembly in two domains, the automotive domain and industrial or factory automation. After that I'll describe a couple of the challenges that we ran into in seeking to do this. The other thing I want to say before we go any further is that all of this work is the product of a large collaboration that is international. We had folks at Bosch research in both the US and over in Stuttgart Germany working on this. We also worked very closely with our research partners at Carnegie Mellon University and we worked with folks at the Bosch subsidiaries Rex Roth and Etos to figure out what it will take to bring WebAssembly code closer to industrial grade. So with that let's talk about how we could potentially use WebAssembly to streamline automotive software. As I've kind of alluded to, automotive software is necessarily complex. You've got all of these ECUs that are working together and you've got that tight coupling to the underlying hardware. Further, within this domain we really, really want some degree of vendor compatibility. As a software vendor, this is coming from the Bosch perspective, we want to be able to reuse software across vehicle generations and across vehicle manufacturers. So if you actually dig into the code that you'll find in a car, you come upon all of these layered software architectures. For instance, AutoZar Classic is a framework that was designed specifically for real-time microcontrollers. That way, if you design your code using AutoZar, you can port it to other microcontrollers running AutoZar Classic. Great, right? The trouble is this doesn't scale as we start to increase the complexity of the hardware. So we had to develop another layered software framework for our vehicle computer class of hardware. This framework is called AutoZar Adaptive. It's also layered, it also offers compatibility with other micro processors running AutoZar Adaptive, but you don't get any more compatibility than that. And this is all still a world better than the bespoke code that's hiding underneath. This is code that's running usually on low-end ECUs, has been highly optimized over the years to hit tight regulatory and cost requirements. So the key is we've got this complicated world of software in automotive, and we're doing the best we can, but from the customer's perspective, all they see is that in their car, software updates are slow compared to other domains, and hardware updates are glacial. So how can we use WebAssembly to fix this? Well, we think that we could potentially use WebAssembly as that kind of narrow waste of automotive software. If we do it right, we can use WebAssembly to fully decouple programming languages and the applications built using them from the underlying schedulers, drivers, operating system. That would let us slot in an application on top of any of the hardware running in your vehicle and any of their associated software stacks. That sounds great. Let's just do that. Well, the trouble is that each of these software frameworks has a very particular way of interfacing down to those lower layers of the stack, and they desperately need to do this to get at the very real data that your car is processing all the time. So the first challenge that we need to overcome here is defining a unifying interface to these underlying software layers, and we need to make sure that we maintain the safety and real-time guarantees of those software stacks in the applications themselves as we do this, because that's a serious concern. There's a lot of value there. This is a big challenge. The one reason that I'm excited that we can potentially do this is that my colleague at Etos has developed a prototype version of this. He deployed a WebAssembly application on top of an open-source automotive stack and was able to pretty quickly actuate a real motor. The key here is that he leaned on the Coveisa project. This is an open-source automotive community trying to make it easier to develop automotive software and to deploy it, primarily by leveraging modern software engineering practices, and he used a number of different projects from the Coveisa world in this stack, but I think the most important of them was the vehicle signal specification. This basically gives us a uniform way of talking about data in your car from the perspective of an application. It means that we're all talking about speed the same way. It means that we're all addressing the right front vehicle tire pressure the same way and gives us a degree of uniformity. From the vehicle signal specification, my colleague then pulled in WebAssembly interface types and the associated wit tooling. By pairing wit with the vehicle signal specification, he was able to very quickly generate the stubs he needed to actuate a motor from WebAssembly. So from that perspective, I think the vehicle signal specification is a good starting point for this unifying interface to the underlying software layers and the data we need. The trouble, though, is we still need to address this point, which is how do we maintain the safety and real-time guarantees of this system. So to do that, I actually want to switch domains a little bit and see how we're doing this in the industrial automation industry. Because the key here is that industrial automation, like automotive, has tight real-time and safety requirements. So the first thing to know about an industrial automation system is that, like a car, it is also, these networks tend to be rigid. What you'll see are a series of programmable logic controllers or PLCs. I'm specifically featuring the Control-X PLC made by our partners at Rexroth in this little diagram. The new thing here is that in the industry 4.0 kind of movement, what we're trying to do is add in a little more compute power to these networks by adding on a couple of edge servers. The key trouble here, though, is that we've added in these edge servers, but it's not like we can just take our PLC applications and run them on the edge server to say have them run faster. That's not how these applications work. So if we want to achieve this degree of flexibility, there's a couple of things we need to do. The first of those is establishing some sort of uniform communication interface. I need to be able to access the same data from anywhere in the system if I want this level of portability. The next is I need some real-time aware orchestration. I can't simply move an application from one controller to another if my network is going to get in the way of the real-time constraints that I need to hit, so I need an orchestrator that knows that. And finally, I need a form of ultralight weight virtualization so that I can actually run my programs on different platforms, and that is where WebAssembly comes in. So to bring this idea to life, at Bosch Research, we define this architecture that we call SilverLine. SilverLine leverages WebAssembly in order to achieve this degree of flexibility and kind of tick off those three things that we need. So the key features of SilverLine are a published subscribe bus that gives us a very simple programming model to build on, a real-time aware orchestrator that we built that uses WebAssembly modules in order to monitor the status of each of the nodes in the network, and finally, we built the SilverLine runtime, which is basically a thin layer below WAMR, an open source project from the folks over at Intel, thank you so much, that lets us run WebAssembly modules, and we're able to deploy this on both our PLCs and our edge server to get uniformity throughout our network. Now, this is kind of the idealized SilverLine architecture. This is what we use when we can rebuild the whole system from scratch, but that's not particularly realistic. What we want to do is be able to run WebAssembly modules alongside all of this legacy code that folks have spent years building. So in order to see what it would take to do that, we partnered with folks at Bosch RexRoth to build a stack that would work on top of their Control-X code. So what that stack looks like is shown here. Effectively, on top of the programmable logic controller, they use a real-time Linux core, a container host to host snaps, and then the Control-X data layer. This is their version of that uniform computing layer so that they can access the same data anywhere in the network. So that's where we hook in. One of the researchers at Bosch built what we call Waxi, the WebAssembly Control-X interface. This is the interface that we need to go from WebAssembly modules all the way down to Control-X data layers and connect to very real sensors and actuators. So Waxi done. We made a handful of modifications to our SilverLine runtime, slotted in WAMR unmodified, and then we're able to run control, I-O applications and data processing applications all on our WebAssembly modules interfacing to PLCs. So what did we do with that? What we showed is that we could trivially offload the control part of an industrial control application to a remote edge server without having to change the code. And we'll see that in, hopefully, on this video here. What you're seeing is the industrial automation lab in Stuttgart, Germany. The main idea of all of this is that we use industry-grade hardware running a top industry-grade software to perform an industry-grade demo application. This demo application is an inverted pendulum. It's this inherently unstable system that you need to hit really tight real-time limits on in order to keep the inverted pendulum upright. So what you'll see here is that we've fully offloaded control using WebAssembly to our edge server and were able to keep that pendulum upright. So that was exciting. It added flexibility to our system, but hitting real-time guarantees isn't enough in this domain. We also need to make sure that we add in a degree of fault tolerance now that we offloaded control of our application. So to do that, we connected two edge servers to our PLC and deployed WebAssembly control modules onto each of them. From there, we also deployed two monitoring modules, outlined in yellow, that are keeping track of whether or not that node is connected to the network. Then we'll have one server be kind of our primary, the one that's actually talking to the pendulum, and should it fail, the backup will take over. So we'll see that happen in this video. What's basically going on at this point is that with a touch of a button, my colleagues deploying a series of WebAssembly modules to both of these edge servers. The pendulum is shown standing upright in that circle, and my colleague is disconnecting the edge servers. The key here is that we're able to disconnect and reconnect these servers without the pendulum ever falling down. That means we achieved a degree of fault tolerance so that we can trust the system a little bit more. And this was made much, much easier because all we had to do was deploy a handful of WebAssembly modules. So with that, I've given you just a taste of what we were doing with adding flexibility into industrial automation systems. And before I run out of time here, I want to talk about some of the challenges we ran into in bringing WebAssembly to cyber physical systems. So the first thing we ran into, particularly from the fault tolerance side of things, is that we're not seeing widely adopted checkpointing mechanisms in WebAssembly. Certainly there are a couple of runtimes that are kind of one off and doing checkpointing, which is great, but we don't see a widely adopted kind of standard for how you would do checkpointing that would be easy to implement across multiple runtimes. And this is important because what we did in achieving fault tolerance was really suboptimal. We used what's known as a hot standby, just having another module on that other machine kind of lying in wait. But WebAssembly spins up fast enough that if we had a checkpoint from the machine that failed, we could simply call up a new WebAssembly module on the awaiting server in order to achieve fault tolerance. So what we need here is that basic definition of, okay, what's the minimal sandbox state required to do a reasonable checkpoint? What's the minimal runtime state? What's the minimal system interface state that we would need in order to achieve what's known as a cold standby and reduce that overhead? The next thing we ran into is that porting applications from so that they run from the cloud all the way down to low end ECUs still means that you're limited by what the smallest and slowest devices can achieve even if they're all running WebAssembly. So our low end ECUs here really can only support minimal wasm runtimes and system interfaces. That meant when component model tooling came out in wasm time, we couldn't implement that quickly in our system because our low end ECUs can't support it. So I think from the embedded perspective, what we're hoping for is as wasm continues to grow to be able to have our cake and eat it too. We're hoping that WebAssembly continues to have this minimal subset that's broadly compatible with many runtimes but is small enough to run on tiny, tiny targets. Now, the final challenge I want to point out is really the big thing that's keeping us between taking WebAssembly and deploying it in real safety critical systems right now. And that is that establishing a safety of an emerging standard is extremely challenging. This terrifying V here is all the processes you need to go through in order to sort of safety certify a piece of code to run in a vehicle. And I say this not to make everyone despondent but to point out that a concerted effort to safety certify a wasm tool chain will be required in order to get WebAssembly into safety critical systems. Now, I've been just a smidgen doom for those past couple slides, but I want to point out and make quite clear really that at Bosch Research, we still see WebAssembly as the most promising solution to all of the problems I have listed. For instance, I'm complaining about having to use a hot standby in that fault tolerance application so we can get failover. The key though is that a hot standby in WebAssembly has such a low overhead compared to systems like containers that it's really a cheap decision. We go with WebAssembly. From the heterogeneous targets perspective, as much as it is challenging to make this work, WebAssembly is still our best bet because that low memory overhead means we have more room for applications on tiny targets. It lets us get much more functionality out of these systems. And finally, in terms of safety certification, while I don't mean to downplay how difficult this will be, WebAssembly's formal foundations would seem to put a bound on how much effort it's going to be to safety certify wasm because there's so few unknowns. So from that perspective, we see WebAssembly as the most promising solution for bringing lightweight virtualization all the way across cyber physical systems. So with that, I've given you a taste of what we think we can do in both the automotive and industrial automation domains, and I've talked about some of the challenges that are standing in our way. With that, thanks very much for your attention. I look forward to discussing this with you all at the rest of the conference.