 Hi everyone, welcome to our presentation today called Living on the Edge using IoT devices on Kubernetes WebAssembly applications. My name is Kate Goldenring and I'm a software engineer at Microsoft on the Azure Edge OS team working on an open source project called AUKRI. And I'm Rodrigo Rodriguez and I'm a software engineer at Facebook. So to start us off, let's talk about what we hope you get out of this presentation. We're going to demonstrate the current ability of WebAssembly applications to discover and use IoT devices on Kubernetes. We also hope to illustrate the delta between what the WebAssembly systems interface supports today and the potential of what it can support in the future as it matures. And then finally, we are going to show how AUKRI simplifies IoT device discovery and use in Kubernetes. So to break down those goals into an agenda, we'll start out with discussing an IoT scenario that we hope to enable. And then we'll talk about the steps that we went to enable it. So the first being implementing device plugins in Creslit and the second being porting AUKRI from being a containerized application to a application of WebAssembly modules. And we'll discuss some of the challenges that we encountered with that. Finally, we'll just show the outcome in a demo and discuss next steps. So let's talk about that IoT scenario we want to enable. And as we discuss it, we're going to talk about how it can be improved by moving it to the edge, further improved by adding AUKRI and then even more so by using WebAssembly. So let's say that you're a farmer. And an issue you have is that you have a bunch of chickens and there are some coyotes that keep coming in and eating your chickens. So you decide that you're going to install some IP cameras and set up a machine learning application in order to identify those coyotes and alert you to go out and scare them and get them to go away. So the first thing you do is you set up your IP cameras and you're going to go ahead and do this in the cloud. So you deploy some frame server pods in a microservice approach where you're deploying one pod to each camera and that frame server is grabbing the frames from that camera and passing them off to some inferencing pod, which then does the work with the GPU and some ML to identify those coyotes and passes that output to a final alerting pod that tells you to run outside and get rid of the coyote. So this all works, but a lot of farms are in very remote places. And so you start experiencing some latency and so you decide to follow the laws of data gravity and process data closer to where it originates. And you decide to install a server on the edge on your farm. And you're optimistic about your farm and how it's going to grow over time. So you go ahead and add it as a node to a Kubernetes cluster. And so we can see here that we're doing all the processing locally on the edge and we're only sending up the final alerting data to the cloud. On the side here, we're going to keep track of all the steps that you have to do as an operator. So it starts off the same. We're going to deploy three frame server pods, one to each of our IP cameras. And if we look more at this in detail, what you have to do is you have to configure each one of these pods with the connection information for each one of those cameras. Then you're going to go ahead and deploy your inferencing pod. And you have to configure that inferencing pod to not only find the GPU, but also get the output from each of those three frame servers. And then you pass the output from that inferencing pod to the alerting pod. So this is all doable today with Kubernetes on the edge. However, unlike in the cloud where the environment is very homogeneous, on the edge, the environment's heterogeneous and things are changing all the time. So what happens if one of your cameras goes down? Well, your frame server pods idling or maybe erroring out, you have to go send someone out to either fix the camera or you have to bring down that pod. What happens if you scale up your IoT deployment and add more IP cameras? Well, you need to manually go install and pass that preconfiguring and preconfigure each one of those frame servers to connect to the specific camera once again. What happens if you add a node to your cluster and move it on to the next field? Well, you're going to have to go ahead and deploy that inferencing pod again and have it send that information to the alerting pod. So as the environment continues to change, you have to keep interacting with this cluster. And this is where Aukri comes into the picture. So Aukri can be read as an acronym that stands for Kubernetes Resource Interface, and that's because it aims to abstract away the details of device discovery and use. So let's look at how this scenario simplified using Aukri. So the first step now is to deploy Aukri, and you can do that with Aukri's home chart. Next, you're going to install or deploy an Aukri configuration, which is one of Aukri's custom resources. And this is how you tell Aukri what you want to discover and what you want Aukri to automatically deploy to what it discovers. So in this case, specifically, you specify a discovery handler, and Aukri supports three discovery handlers, ONVIF for discovering IP cameras, UDEV for devices in the local Linux device file system, and OPCUA for industrial machinery. So in this case, we're discovering IP cameras, so we're specifying the ONVIF discovery handler. Next, we are going to specify that for each camera that's discovered, we want our frame server pod to be deployed, and that's where we're putting that in the broker pod spec. And we call the workload that's automatically deployed to discover devices a broker. So once you have applied your configuration to the cluster, you can see that Aukri automatically discovered the cameras and deployed those frame server pods to each individual camera, and it injects all the connectivity information into each of those pods. Aukri also automates the creation of services, so you can see here that it created one service to provide a stable endpoint for getting access to the frames from all of the cameras. So now that we've discovered the cameras, let's go ahead and discover our GPU with Aukri. So we're going to apply another configuration. This time, we're going to use the UDEV discovery handler, and we're going to specify the inferencing pod as our workload. After deploying that, we can see that we discovered our GPU. So we're back at the previous first stage of our last scenario, but now let's see what happens as the environment changes. So we scale up our IP camera deployment, Aukri automatically discovers those cameras and deploys more frame servers. We add a node, Aukri automatically discovers the additional GPU and deploys another inferencing pod. So we can see here that as the environment continues to change, if there's intermittent connectivity of devices, Aukri will bring down pods. When they come back, Aukri will bring them back. So Aukri handles these unique scenarios on the edge. Another thing that's important to Aukri on the edge is being as low footprint as possible. And that's because oftentimes the compute on the edge is very constrained. So these nodes could be Raspberry Pis, or they could be Intel Nooks, for example. And so because of this, Aukri was written in Rust to make as small binaries as possible. But with a lot of the innovation that's been happening in the WebAssembly space, with the WebAssembly systems interface, we thought, could this be even more optimized if Aukri's components were WebAssembly applications instead of containers? And that's why we're here today at Wasm Day at KubeCon is because we wanted to present an exploration that we did to kind of try this out and see what would happen if we went about converting this from a containerized application to one of WebAssembly modules. So the first question when exploring this is, first, is this possible? And the second is, why would we do this? So for the first question, we know that the WebAssembly systems interface provides extensions to WebAssembly that enable running WebAssembly applications in cloud contexts, such as right on the server. So we know we can run our WebAssembly modules on servers on the edge. Furthermore, we know we can run WebAssembly modules on Kubernetes due to a lot of the work that Crestlet has done. Crestlet is another DEIS Labs open source project that is a Kubernetes kubelet written in Rust, and it allows for running WebAssembly modules on Kubernetes just as you would containers. So we know we can do this. Now the question is, why would we do this? Well, WebAssembly modules not only have a smaller footprint than containers, but they're also highly efficient, have low memory consumption, fast boot up time, and sandbox by default. And all of this can be summarized in this comparing contrast grid of containers versus WebAssembly modules that are Wazzy compliant. And you can see here that some of the red on the Wazzy side are the fact that Wazzy WebAssembly modules are only able to be single threaded and they have limited network access. But one thing to note here is that all of this red is due to the fact that the WebAssembly systems interface was only created two years ago. And there's work underway to turn these red blue and enable these features. And so we expect that over time, everything on this Wazzy side will be blue while all the downsides on the containers aren't going anywhere. And so really excited to proceed with this POC of making this happen. And we know that any of the challenges that we encountered will go away over time as WebAssembly systems interface matures. And this can be summarized in if Wazzy existed in 2008, you wouldn't have needed to create Docker. That's how important it is. WebAssembly on this server is the future of computing. And that's a quote by Solomon Hikes, the creator of Docker. So he's also acknowledging the power of using WebAssembly modules in some cases. So with that being said, this is the picture that we want to paint. And in order to create this picture, we went through two implementation steps. The first of which is we had to add device plug-in support to Cresslet because Ocree requires it. And the second was we had to convert Ocree's containerized components from containers to WebAssembly modules. And so I'll talk about the first. And then Rods is going to jump in and talk about the second. So before we talk about how we implemented the device plug-in manager in Cresslet, let's first talk about what the device plug-in framework is. So out of the box, Kubernetes has several compute resource types, namely CPU, RAM, and huge pages. And this is why Kubernetes is powerfully declarative because you can apply a pod to your cluster and specify that it needs this much CPU and this much RAM in order on the node to be reserved for it in order to run. And so you know that that workload is going to have exactly the resources it needs. The device plug-in framework takes that one step further and allows you to advertise other node-level resources to Kubelet so that then they can be requested in a pod just as you would compute resources. And this was really motivated by GPU access. You can see here that we have a GPU device plug-in that's advertising that as a resource to the node. And then we have a GPU pod that's requested it in this container spec on the side. And Ocree uses this framework for when it discovers devices, it then creates device plug-ins to represent them. And thereby, when it automatically deploys applications, it requests those resources. And so that's why we needed then to implement the device plug-in framework in Crestlet. And so I got the opportunity to take on this work. And what this meant was basically reverse engineering and porting the traditional Kubernetes Kubelet device plug-in manager, which is written in Go, and implementing that in Rust in Crestlet, which is a Kubernetes Kubelet written in Rust. And what this work entailed was implementing the registration service for device plug-ins in Crestlet, and also making it so that Crestlet updates the node spec with that available resource when it's a device plug-in registers. And then finally, that the Crestlet reserves that device plug-in resource for the requesting container when it is scheduled to the node. And then the step beyond this, and what's really cool about the device plug-in framework, is that a device plug-in can specify exactly what environment variables it once injected and what files it needs mounted to any pod that requests that resource. And so this means that now we have, just with this work, a scenario where WebAssembly modules can declaratively request devices on Kubernetes clusters, and the Crestlet will mount any of those files that are needed in environment variables into the WebAssembly module, or thereby grant access to the WebAssembly module for those files. So we understand now how we went about implementing the first of our two steps. So we're going to transition and talk about the second of our two implementation steps to get to that exciting picture that we saw earlier. But before we talk about how we changed Aukary's architecture to have WebAssembly components, let's talk about what Aukary's architecture consists of. So the first component in Aukary's architecture that we've seen earlier is Aukary's first custom resource definition, which is our configuration. And in a configuration, you specify what you want to find by specifying what discovery handler you want to use. We mentioned ONVIF and UDEV and the third OPC way earlier. And the second part of configuration is the broker pod spec, which is where you specify what workload you want automatically deployed to discovered devices. Once you apply that to your cluster, the Aukary agent, which runs on every node in the cluster, will see that, and it will tell the appropriate discovery handler to start looking for those devices, whether locally on the node or across the network. For every device that the discovery handler discovers and reports back to the Aukary agent, the agent will create a device plugin for it, thereby advertising that device to the cubelet and the cluster at large as a requestable resource. The agent also creates our second custom resource, the Aukary instance, to represent the device in its usage. So we now have this inventory of the devices that Aukary has discovered. And finally, our last component is the Aukary controller. What it does is when it sees an instance, it will automatically deploy the workload that it was specified in the configuration to use that instance and request the device plugin that was created by the agent, thereby deploying a pod that has all the resources and connectivity information that it needs to connect to that device. So with all that, I'll pass it off to Rod to talk about how we change this architecture to contain WebAssembly components. Thanks so much, Kate. So yeah, now let's go a little bit in depth on what is some of the change we have made to bring up with the WebAssembly environment. You can see right there on the design, on the updated design part, the upper part of the design is actually 100% the same as Kate was showing up on the previous slide. So we didn't change anything there. So you can still have the configuration, the instances, the correct CRDs, and the Aukary controller is still there. So the core change comes on the node level. You can see that the most impactful change is it's now, every node will be a crosslet node. So it's not running on a normal cobalt anymore. And what this means is this CRDs node is capable of running WebAssembly workloads. So this is what we're going to be aiming for during this node. So we have WebAssembly application running on each node. And the other key difference is right there on the Aukary agent. Not that the Aukary agent is different, but it's now instead of running inside Kubernetes, it's not running locally. And this is because since CRDs is only capable of running WebAssembly workloads and the Aukary is a container application, it was required to be removed from Kubernetes, but still they're communicating a lot with the rest of the Kubernetes cluster. It's not deployed to the node yet. And the rest is, of course, the more interesting part, which was the WebAssembly application. So we have this core handler and there was a broker. So for this POC, we have used debug echo to the core handler. And debug echo is a debugging Aukary discovery handler. That serves as a test for Aukary discovery process. So it's discovery fake devices and start using them on their applications. So you can see that the Aukary is really a simulation of the Aukary process. And of course, the broker pods that are now WebAssembly applications. It is still deployed automatically by the Aukary controller. So this function is still the same, but now WebAssembly application interacting with the devices. So now let's go to the next slide, where we see some of the technical limitations that we have managed to overcome for doing this design. Right down the left, you can see that this was our ideal design, the one that we managed to, the one we wanted to do, like if there was no limitations. But of course, Katie was talking before how WebAssembly has a single thread and it has a limited network access. And turns out that the Aukary agent is a communication the discovery handler via GRPC. And since GRPC requires sockets and HTTP2, which is not supported by WebAssembly yet, we had to create this new component called the GRPC proxy. And what this proxy does is it mediates the communication between the Aukary agent and the discovery handler. And it does that by using the file system. So the GRPC proxy will connect to the Aukary agent and you'll send everything that the agent sent to an input file that can then later be written by the discovery handler that can start the discovery process. And as soon as the device has been discovered, it will then be printed to an output file. So this way the agent will know about the device has been discovered and that can start the continuum of the Aukary process. So now let's go for the demo. You can see right down the right, we have our updated design with the overcomes that we have to do it. So our demo will be composed of four steps and I'm going to be breaking down them before we jump to the video. First, we're going to be setting up our cluster. So we're going to be making sure we are running a two-note cluster, one being a normal cobalt and the other one being precedence. Then we're going to be deploying Aukary. So what this part will be composed to is we're going to be doing the Helm Solution Work Factory. You're going to be seeing this really similar to the one we are really used to. And we're going to be running the local programs. So this being the GRPC proxy and the agent will be also required to be running locally there. Then we're going to be applying the Aukary configuration. Sorry, the Aukary debug echo configuration. So we can make sure that the Aukary agent already know which device to look for and also configure the broker positive. So the Aukary controller can deploy the correct broker as soon as they find the device that was created was discovered, sorry. And finally, the final step will be to deploy our discovery itself. So we can start to see the discovery process taking place in full action during the demo. So yeah, now let's go for the video. And for this, we are using the Kubernetes dashboard to better visualize the pause and the broker and the cluster situation in general. So let's go, let's start. And first, we are going to be setting up our cluster. So you can see that right now, we only have one node, our normal Kubernetes Kublet one and that will be serving as a control plane for us. So now let's deploy our cluster node. So now let's run the cluster command. You can see that this is really similar to the one we are really used to doing cluster. Now let's approve the certificate. So we can see that the cluster node has the right permission to handle Kubernetes. So yeah, right there, you can see that we are now with a two node cluster. What this means is our cluster is ready for receiving WebAssembly workloads there. So now let's go for the deployment of Aukary. So for this, we're going to be using this ham installation right there. And you can see that this installation is really simple, there's only two lines. And it's really similar to the one we are always used to doing installation of Aukary. The only key difference right there is on the last line where you can see that we are setting the agent to be disabled. So what this means is we do not deploy the agent to any pod. And this is because as I said, we're going to be using our local running agent. But yeah, now let's apply it. So the correct CIDs and the Aukary controller should already be deployed during this ham installation. So now let's go back to the dashboard and show the new pod that we have created. So right there, we have the Aukary controller already running on the control plane node. And we have the two CIDs right there. So the configuration and the instance, there is still no configuration yet. It's going to be still waiting for one. So now let's go back to the terminal and deploy our local programs. The first one I'm going to be using is the Aukary agent. Nothing really stands up on that. And the last one is the GRPC proxy. So this GRPC proxy is where we started to see if something's interesting and different than Aukary itself. Right down the second line, you can see that we are specifying the discover handler name to be debug echo. And what this is doing is since this proxy was designed to be to work on any web assembly discover handler, we need to specify which device this GRPC proxy is responsible to. So the Aukary agent knows that this proxy is able to find debug echo devices. So as soon as the Aukary agent knows it needs to find this type of device, it's nowhere exactly where to look for. So now let's run it. And as soon as we run, you can see that of course it's not, you can see that right down the Aukary agent, it's already registered. So the Aukary agent is already already this new discover handler, sorry, this new GRPC proxy is there. And so now let's apply a configuration. And this is where we start to see things interesting and a little bit changed then after itself. So as soon as we run this configuration, we go back to the dashboard and see more details on that. You can see that it's already showing up there. And the things we're deploying to this configuration is the same format as Kate was mentioning before. So the after configuration is composed on the broker pod spec and the correct configuration for the discover handler. And first talking about the broker pod spec, this is where we start to see things different because it's now deploying to the, it's not a web-assigned application. So it's not a container application anymore. It's still going to be deployed automatically by the acrocontroller. But as you can see right down the node selector, we are specifying the Wasm32 Wasi architecture. So this is going to be deployed to the Crescent node. And the image is the OCI normal web assembly image. So now let's go right down the bottom part of the configuration where you can see the discover handler configuration. And you will see it right there. We are specifying the debug echo. So as soon as we apply it, the agent is already looking for the debug echo device. But since we haven't, we have not a discover handler yet, it will keep waiting for one. So now this is what we're going to be applying now. So if you'll go back to the pod section, you can see that we have already a discover handler there. So at any moment now, a new device was discovered. And right there you can see the device was discovered and the acrocontroller acknowledged and has already created a broker pod for that. And right down the acrocontroller instance part, you can see that we have a debug echo device. So now let's go back to the terminal and we're going to be seeing a more general cluster situation for acrocontroller components. So we're going to be seeing the acrocontroller instance that we have just discovered, the acrocontroller configuration that we have applied some moments before. And right down the pod selection is where we get to see the more interesting thing. Because right down the things that I'm highlighting right there, the acrocontroller, deployed this broker pod. So this broker pod is a WebAssembly application that's interacting with the device. And it's doing this using this, that we're going to be showing right there. This is requesting this pod to have a WebAssembly, so to have a device named exactly as this in order to run. And what this is doing is it's using the proper device plugin manager that Katie was showing up before. And this is what we're going to be, this is what we need to link the WebAssembly application to the device. So now let's continue on. And now that we have showed the describe of the pod, now let's check the logs and see what's doing with the device itself. And you can see it right there, the pod is running and using the bug activity as the info zero. So what this demo really shows is a WebAssembly application discovering the device and then after controller deploying automatically a application that's using this device. A WebAssembly application that's now using the proper device that was discovered by the WebAssembly application. So now let's continue on with the presentation with some takeaways from that. And of course, the whole point of this experimentation and this study was to reduce the African fingerprint so we can work on even more constrained environments. And as you can see right down this slide, we have to reduce the OCI image is now 54 times smaller than before went in from 103 megabits to 1.9 megabits. So I don't know about you guys but this is a really domestic decrease there and really shows a sense of how WebAssembly is on a completely different scale than normal containers in terms of memory usage. And right there, you can see that we have a more in depth look at where this improvements come from. So WebAssembly binary itself is way smaller than Rust and Rust has already a really small binary itself. So WebAssembly is even better than that. And we have also reduced the container. So by removing the container, we are also reducing a really major part of the memory consumption. So now let's go for a roadmap. And of course, the first part and the more immediate one that we can start working on right now is to create a real discovery handler. So we can make sure that this POC work for example on a scenario that Katie was talking before with the cameras. So we can start to see this application on a real problem, a real environment. So of course, Acre itself has some, already some discovery handler in place. So maybe this test will be to convert this Acre discovery handler to be a WebAssembly application. And this of course is where we need everyone's help. So everyone's watching everyone that's, me, Kate, everyone that's on the community. We can start doing this right now so we can make a even more powerful environment on the WebAssembly edge community. And the next two bullet points is something that we as a community is still working on hard. That is trying to solve some of the limitation of WebAssembly right now. So we're going to be adding sockets to Wazzy and the multi-trading support. So this is both something that we need. There is still a lot of progress already going on the community, but this is something that we really require to create even more complex application on WebAssembly. And finally, our final goal would be the Kubernetes plus-plus-plus part where we want everything to be running on Kubernetes. So for example, even the Acre agent that is a really complex application that has a lot of network access, a lot of things running at the same time can also be deployed to Kubernetes as a WebAssembly application. And yeah, so to conclude this things up, if you want to get involved, there's a plenty of ways to do so. This PLC is right down the first link on the first link on this slide where you can select the demos and everything that we have shown up during this demo. But of course, you can also get in touch with Acre, Cursuit and the Wazzy specification itself. So yeah, thanks so much for attending. Thank you all for coming. We are going to be here for a 15-minute Q&A and we're excited to answer any questions you have.