 Hi, everyone. My name is Kate Goldenring. My name is Yuri Apple. And we're both software engineers at Microsoft working on an open source project called Aukri. And today we're going to talk about how you can use Aukri to make IoT devices accessible to your Edge Kubernetes clusters. So we'll first start off by talking about the motivation behind Aukri. And then we'll look at an IoT Edge scenario and how it can be simplified using Aukri. Then we'll dive into Aukri's architecture and look at one of Aukri's latest enhancements that make it more easily accessible. And finally, we'll talk about how you can get involved. Okay, so a bit of motivation. A typical cloud Kubernetes application has a fixed set of dependencies. These are other cloud services that the application depends on for their functionality. And these services are generally highly available. On the other hand, an Edge application depends on variable number of data sources, especially an Edge application that does any data processing. Now, these data sources, they might not be highly available because they might be represented by a simple devices such as IP cameras, temperature sensors. Your Edge application can also do control such as control of robotic arms. And again, these are not highly available sources. So then the question is, how can your application discover these when there is a variable number of them? We cannot put these devices to be part of a Kubernetes cluster because many times they are single-purpose devices, they have a small footprint, or they have not been certified to run Kubernetes. And so how can your application take a dependency on them? When we looked at this, we couldn't find a standardized way to achieve this, and that's where we came up with Aukri. Aukri exposes the individual IoT devices into Kubernetes clusters extended resources. This then allows your application to take a dependency on them similarly like you would do on any GPU where you can request access to the extended resource. Aukri itself, the name is an acronym, it stands for Kubernetes Resource Interface for the Edge, and it also stands for Edge in Greek. Now, Aukri works in two phases. First, it discovers and advertises the IoT devices to the cluster. It then monitors the availability of those devices to make sure if a device disappears, maybe it's disconnected or starts malfunctioning, it stops advertising it to the cluster. Second, this is an optional piece. You can request Aukri to schedule a workload when a new device is discovered. This allows you to dynamically, in a plug-and-play fashion, schedule your workloads if and only if the devices are available. Now, to achieve this, Aukri has an extensibility point called discovery handlers. We don't think that any project can handle discovery of all IoT devices. There is just so many protocols out there. And so the discovery is uploaded to the discovery handler. And you can take a dependency on several discovery handlers that we have already available with Aukri. We support OnViv for IP cameras, Udev for locally attached devices, and OPC UA for industrial settings. But you can also bring your own or take a dependency on any other discovery handler coming from a community. Aukri itself is purely open source project. There is no Microsoft hidden source in it. It is released as part of Deus Labs, side-by-side with other Deus Labs projects such as Helm. Aukri has been built in Rust, but it doesn't mean that the device discovery handlers needs to be written in the same language. You can pick any language of your choosing. So now let's look at a typical IoT scenario and let's see how Aukri can help us. Say you're building a smartphone. You will care for your crops and for your animals. For your crops, you might want to deploy some soil moisture and pH sensors, some temperature sensors. For your animals, you might want to deploy some IP cameras and microphones to ensure there are no predators attacking your animals. So in order for you to take advantage of all these sensors, you need to have some application doing the data processing. Let's zoom in specifically on the IP camera scenario and see how you could design your application. First, we'll go over the scenario without Aukri to understand better the complexity involved. Say you start with your IP cameras deployed and you need to deploy the application doing the processing. We'll take a simple microservices-based architecture where we have a frame server pod running for every camera. It's processing the RTSP stream extracting individual frames and passing it onto the inferencing, which takes advantage of GPU and ML model to discover any predator. And when a predator is found, it sends an alert. So great, except for the fact that we are running on a farm, not in a data center. And so we might not have the connectivity and the bandwidth necessary to push full video streams or multiple video streams into the cloud. And so instead, the data attracts compute data gravity. So we can deploy the Kubernetes cluster onto the edge. It's a simple Kubernetes cluster to begin with, with the single GPU. We have our alerting pod running in the clusters. And that's okay because that produces a low bandwidth stream of data that we can push onto cloud for additional processing. For fun, we'll also keep track of how many steps an operator needs to do to provision all the PCs into the edge cluster for your application. So we have the IP cameras. And in the microservice fashion, we'll deploy the frame server pod per camera. And we have to manually provision the configuration, the connection string, so that the frame server can start pulling frames from its assigned camera. As we have that, we need to provision an inferencing pod with an ML model and attach it to a GPU so that we can start doing processing. This inferencing pod needs to understand what are all the frames that are assigned to it, so it can pull the frames and start processing them. And then when the processing is complete and the predator is discovered, we send an alert the same way as we did in the cloud time. Now, what happens if a camera goes down? So maybe say it goes disconnected or it's malfunctioning. Well, now you have a frames server pod sitting in your cluster and either idling or it just keeps restarting. That's not great, just consuming resources. So you might need to send an operator to go and remove it from your cluster. Next, say you want to scale. Well, now you need to again go schedule multiple frames reports provision then individually with the connection string to the cameras so that they can pull on the individual frames. Say you want to scale your cluster. Okay, you deploy a new node. You need to again go and manually deploy the inferencing pod to take advantage of the other GPU. What if any node goes down? Well, maybe you repartitioned, maybe you repartitioned, you will need to repartition which frames server pod goes to our process on which GPU. And so there's a lot of manual steps you kind of had to go along the way in order to create an application that can react to changes in the physical environment surrounding the cluster. So let's see how we can simplify this flow with ACRI. We'll start with the same set of cameras will deploy the same Kubernetes cluster, but but we're going to deploy ACRI onto this cluster. That's the green box. Kate will later show us what are the pieces of ACRI but for now let's focus on high level. So with ACRI in place, instead of manually deploying the frames reports will deploy ACRI configuration. This is a custom resource, which allows you to declare at every state. What are the devices you want to expose into your cluster. It's composed of two main pieces. So first, we request a discovery handler, and we are using here on with to discover IP cameras. In practice, there will be additional configuration such as filters. And then second, you specify in for the plug and play part you specify the broker you want to deploy that will take an event that will expose the device into the cluster. Now, in our case that's the frame server pot. This broker will be automatically provisioned with the configuration to access the individual cameras. In our case it's a RTSP URL. Furthermore, ACRI also deploys a service that's the blue box sitting behind the frame server pots. And this aggregates and load balances across all broker pots that are deployed on behalf of single ACRI configuration. This allows us to access all the frames of reports in a uniform fashion. Next, we need to take advantage of the GPU we need to do the inferencing. So, similarly, we can use ACRI in this case are you deaf discovery handler to detect the GPU and when the GPU is detected to deploy the inferencing pot. In this case, say again, if any camera goes down, that's okay, ACRI will discover that the device is no longer available, and it will automatically evict the frames of report. If you want to scale. All we need to do is we just connect the IP cameras, ACRI will automatically discover them provision the frames of reports configured correctly, and our inferencing solution didn't have our application didn't have to be changed in any way on from the We want to scale the cluster we add a new note ACRI will automatically detected because we have provided the GPU configuration deploy new inferencing pot and again application scale in completely Kubernetes native way without the operator having to go and make changes based on the changes that happen in the physical environment surrounding the cluster. So, with that, I'll transition over to Kate who's going to walk us through the details of the green box. Awesome. Yeah, let's dive into this green box that's ACRI's architecture. So, ACRI is Kubernetes native it's made completely of Kubernetes components. The first of which we saw earlier is ACRI's configuration. And that's where you tell ACRI what you want to find. And you do that by specifying what discovery handler you want to use to discover devices. So this could be on VIF like we saw earlier for IP cameras, Udev for local devices such as GPU, or it could be OPC UA for industrial machinery, or it could be your own custom discovery handler and we'll talk a little bit more about the development life cycle of creating your own discovery handler later. Also in this configuration you specify what custom workload, if any, you want deployed automatically to discover devices. We call this our broker. So once you apply your configuration to the cluster, the ACRI agent will see it, and it will tell that discovery handler to start looking across the network or lowly on the node for those devices. For every device that discovery handler discovers, the agent will expose it to the cluster using the device plugin framework and also create our second custom resource, the ACRI instance to represent it and its usage. The controller will then see this instance. And if a broker pod was specified, it will automatically deploy this broker to the discovered device and inject in it as environment variables all the information it needs to connect to that specific device. So for example, if these were IP cameras were discovering, maybe that would be an RTSP URL for the specific device. Let's look at the flow of how you could use ACRI. And in this case, say we have some custom application that wants to utilize the frames from IP cameras that are around the cluster. So the first thing you need is a Kubernetes cluster. So here we have one with a control plane and two worker nodes. And it has all the Kubernetes components already a part of it. So at this point, we can go ahead and deploy our custom application that's going to benefit from the work that ACRI is going to do. And we'll say that this custom application has been pre-configured to point to a service that will automatically be created by ACRI. Now we're ready to install ACRI. And ACRI can be installed using its Helm chart. It's just a simple Helm installation. So once you do that, you see all of ACRI's components that we saw previously on our architecture slide pop up. So we have our controller running in the control plane and our on-vift discovery handler and agents running on each worker node. At this point, we can tell ACRI to start discovery by applying our configuration to the cluster. In this case, because we want to discover IP cameras, we're going to specify that we want to use the on-vift discovery handler. And since we do want workloads automatically deployed to the discovered devices, we're going to specify the image for our workload. And in this case, we'll say it's a frame server like we saw in our previous scenarios. So this load will connect to a camera, grab frames from it, and serve those frames over some GRPC interface. The last thing to know about this configuration is this capacity setting. That's where you tell ACRI how many nodes can you utilize a device at once. So here we're going to say only one can deploy a broker to use this device at a single time. So once you've applied your configuration to the cluster, the ACRI agent will tell the on-vift discovery handler, hey, start looking across the network for these IP cameras. We don't see any IP cameras right here on this slide, so let's go ahead and connect a camera. So this could have been done before deploying ACRI, but let's say that you do it now. So the discovery handler will see this camera and tell the agent. And the agents will create an instance to represent this camera. The controller then sees that instance and automatically deploys our broker pod to one of the nodes that could see this camera. It also automatically creates a service for all of the frames from all of the cameras. And that's the one that the custom application was preconfigured to point to. So at this point, the application is doing the work it was intended to do. It's getting those camera frames and doing whatever it wanted to do. But what happens if this node were to go down? Well, since the other node said that it could also see this camera, the controller will notice that the node went down and automatically reschedule the broker. So that once again, that camera is being utilized. However, there was a little bit of downtime. So in order to create a more highly available scenario, you could have set that capacity number to two when deploying the configuration. And then from the start, the controller would have deployed a broker pod to each one of these worker nodes so that if the node went down, there was no lag and downtime for this custom application. All of that flow that we just saw could have been initiated with one helm installation. So you'll see here, you can install just Helm's controller agent and discovery handlers and separately you can create your configuration YAML and use kubectl to apply it to your cluster. Or you can do that all in our using our Helm template and Helm installation. Let's look at one of Aukary's latest enhancements and that is Aukary's discovery handler model. So our discovery handlers do discovery. So they're the ones who that discover all the devices that are either embedded in your nodes of your cluster attached to it or around the nodes. And these can be protocol implementations like on VIF or OPC way. And it could even be your own proprietary protocol that discovers your own unique devices. Originally, these discovery handlers were embedded in the awkward agent, which meant that if you wanted to extend Aukary to discover even more devices, you had to fork the agent and work within its code, which is all rust to include your discovery handler. This was also this was clearly inhibiting and we heard that feedback from the community. And now our discovery handlers live behind a GRPC interface and we'll look a little bit more at that in detail in a second. What this means now is that discovery handlers are deployed their own pods, and they can be implemented in any language. And they're, they can easily be deployed using our helm charts, you can just specify the image for your discovery handler, and they'll automatically be deployed in a demon set fashion like our agents. So for the developers that may be watching this right now, let's go ahead and look at this discovery handler interface. So it consists of two services, the first of which is this registration service, and that's hosted by the awkward agent. And that is the service where when a discovery handler comes online, it will register with the agent saying, Hey, this is my name. This is where I live. And I discover shared devices such as IP cameras that are visible to multiple nodes or unshared devices such as local USB devices. And the name that a discovery handler registers with is the name that you specify in your configuration. So when you specify you use that discovery handler, when you apply the configuration, the agent will say, Oh, that's the discovery handler that I need to ask to do discovery. And it will pass the discovery details that are also set in that configuration to that discovery handler when it calls discover on it. So we're basically abstracted away these details in our previous slides, but discovery details are how you tell a discovery handler what subset of devices you want to discover. So for you dev we don't want to discover all the devices in the Linux device file system. So the you have discovery handler allows you to specify a you dev rule for filtering. In this case, we want to discover only video devices. So in this review, all of Aukri functionality that we've talked about so far. So Aukri is a Kubernetes resource interface. And just like container network interface abstracts away the networking details from a Kubernetes operator. Aukri aims to abstract away all the details of finding and utilizing IoT devices and aims to be a standard way to connect these IoT devices to your Kubernetes clusters. When hand devices are coming and going so when devices appear it'll create resources to represent these devices. When they go away it'll remove that Kubernetes resource that represents that device. And it can enable a plug and play scenario so that when devices are discovered Aukri can automatically deploy workloads to utilize them. And it also makes it so that multiple nodes can utilize the same device at once, allowing these devices to be shared using that capacity setting that we discussed earlier. It can automatically deploy services for you that point to specific devices or all devices of the same type or configuration. And we currently support several discovery handlers that we've discussed so far, and we have community contributions underway for zero comp discovery for MD&S based devices and co-op discovery for devices in constrained environments. But as we saw earlier, we really are trying to make our discovery handler model as extensible as possible so that community members can easily add more and we can continue to discover even more devices. Another way that Aukri is extensible is via its broker pods so you can specify any image for the workload that you want to automatically be deployed to discover devices. And one way that we see broker pods being used is as protocol translation gateways. So maybe you have Aukri discover some USB video cameras and your broker pod will automatically translate that USB camera into being an IP camera. So what's next for Aukri? More discovery handlers. The more that Aukri can discover, the more powerful a solution Kubernetes will be on the edge. We also want to support more deployment strategies. So currently, Aukri has a microservice approach to utilizing these IoT devices it discovers and we'd like to have multiple approaches there. And finally, and most importantly, we want to grow our community. Aukri open sourced only in October of last year. We're still very young and we're really excited to look to the community for direction and see where Aukri is most powerful. If you find Aukri interesting, you can come and deploy our end to end demo for yourself and try it out. You can also learn more about Aukri by visiting our documentation. Everything we talked about there today and much more is described in our docs. We are available. If you have any issues, please let us know on GitHub, or you can talk to us on Kubernetes like Aukri channel. We are always available there for any thoughts, feedback or help with any issues. We are always open to proposals as well. So if you find something that you think Aukri should do for everybody or even just for yourself, please let us know and as a proposal. We are open to contributions. If you want to contribute, that would be great. Otherwise, you can also come meet us at our community meetings. We have once a month community meetings on Zoom where we can discuss other ideas that you might have. Thank you all for the time. We will be here for a little while longer for those of you watching live to answer any questions. Otherwise, like I said, feel free to reach out to us on any other channels. I hope to talk to you soon. Bye.