 We can just start. Hi, everyone. I'm Rhea Batia. I'm part of Microsoft, but I'm also a core maintainer on virtual kubelet. I've been part of this project since its inception in the beginning. I'm excited to talk to you guys about it. We also have one of our providers, which is Azure's virtual node providers here. Hi, I'm Sarvan. I'm a PM in Microsoft Azure working in the Container Compute Team. We use the virtual kubelet library for one of our offerings, virtual nodes, which I'll talk about today as well. Cool. You control the slides. Okay. Some more things about me, so that's my handle on Twitter if you want to reach out. I have a dog named Ru. She's a rescue puppy from Taiwan, actually. Then here are links, which might not be too useful because you can't click on any of these. But when we put them up on the website, the CNCF website, you'll be able to see it. But basically, that's also my GitHub handle, so if you want to reach out there, you can. This is just a bit more about me. You can also reach out to me on GitHub or LinkedIn as well. I work on the two Azure products called Azure Kubernetes Service and Azure Container Instances. Cool. Virtual kubelet. This is our logo. It was voted on by the community and we decided to do this. I like it because it looks like Star Trek, but fast-forwarding to the next slide. What happened? Why did we open-source this? A long time ago, we had this project called the Azure Container Instances Connector, which was created by Brendan Burns, one of the co-founders of Kubernetes. He had this idea that we should start to abstract the core infrastructure of what people run Kubernetes on. Today, we run it on a bunch of VMs, but customers don't necessarily want to run all that infrastructure or see it. They really just want to see their containers or possibly just their apps. This was the first, oh, look over there. Oh, go here. Stand over here. Sorry. Apparently, I have to be here. But basically, he was like, what if customers, basically customers were like, we don't want to see infrastructure. We don't want to manage it. So the ACI Connector happened. We also released a service called Azure Container Instances, and a bunch of other platforms kind of followed that lead, like AWS, Fargate, Alibaba, Cloud, Elastic Container Instances. So these are all examples of services that just spin up containers on the fly, and they charge it by second, and you get to specify the amount of cores and memory that you want for that specific container. You don't see any of the underlying VM infrastructure. From there, we took the ACI Connector project and realized that we wanted to be able to scale out these kinds of container instances, but we didn't want to make our own distributed system. So we wanted to utilize Kubernetes for scaling and for service discovery and for all of the other good bits that Kubernetes has. So that's when we created virtual kubelet. We also realized that if we wanted to open source it and have a community around it, we should also allow other providers to plug into it. So we really just created this interface into what it means to be the kubelet in Kubernetes. All right, so yeah, we're also part of the CNCF, and that's why we get to do talks like this at KubeCon. So this is an intro talk. And yeah, we're also going towards incubating. So in the next year, we're gonna be gathering some case studies and hopefully one day be an incubating project. And we also just announced 1.0. So basically with us saying that we're 1.0, we're saying that we're stable, we're not gonna change the interface drastically, and that anyone that plugs into our interface can be certain that virtual kubelet on its own is basically production ready. So if you wanna run production workloads on this, you can. We've also revamped the entire repo. So everyone used to be plugged into the same binary. So when we build everything would be one binary, like all of the providers, Alibaba's provider, Huawei's provider, Azure's provider was one binary. We have just split everything out. So now everyone has their own repo. And if you build a virtual kubelet binary, it's specific to your provider. So that'll help with a lot of efficiency things, things like that. And yeah, yeah, all right, yeah. So basically this is a snapshot of what happened about two years ago before KubeCon at Austin. We got together a bunch of people at Microsoft and then we also partnered with a company named HyperSH, which is no longer with us as a company. But we moved the project from being TypeScript to being written in Go. And that's when we released it. So virtual kubets all written in Go. And it was also written by some cool people that were part of this cloud developer advocate team at Microsoft. One was Jesse Frazell. And then there's also Ashley over there, Eric St. Martin, Brian Kettleston. They've written some cool books and they've done a lot in the Go community. So yeah, it was awesome to have them. So diving a little bit into the architecture. So one of the cool things about the virtual kubelet project is the way you use it is just like managing any ordinary Kubernetes cluster. You can manage everything through KubeKettle. And as you know, on a master node, there are a number of components there, API server, SCD and whatnot. But then as a user to interact with the workers, you can just use KubeKettle and then each of the nodes will have a kube proxy in which you can actually talk to your pods and containers and have them speaking to each other. So what we've done with the virtual kubelet is we've created a pod that when it gets placed on a Kubernetes cluster, it registers itself as a node. So the Kubernetes cluster thinks that there is actually another kubelet there and that kubelet represents a node. And that way you can schedule pods out to this virtual node. Since it's not a real node, the capacity of this is actually backed by a number of providers different infrastructure, including Azure Container Instances or Alibaba Elastic Container Instances, Amazon Fargate and whatnot. And going along with what Rio was saying, we want to make Kubernetes more pod-centric. Nodes is a concept right now in Kubernetes, but as developers, one shouldn't have to manage and provision nodes all the time. You want to be dealing with pods as much as you can. Yeah. Yeah, I can speak a little bit to the last slide too. Sure. Yeah, so some things that we've done in the community in general in the virtual kubelet community is we've, just like Shrevan said, about providers. We have a lot of questions around how do we show resources in Kubernetes today? Because in a node, we have a finite amount of resources with providers now. There's this question of does the virtual kubelet project mandate how many cores are exposed to the customer and how fast you can scale up? Because one of our primary use cases for virtual kubelet is scaling up really fast and bursting out into this unknown amount of space, basically, compute resource space. With this, how do we tell providers how they should expose those resources to customers? Because the customer says, I need to scale up to maybe a million containers, but the provider is showing that it could scale up to a million containers. Kubernetes will schedule all of those pods, but that might not be true. So then, as a customer, you might only get 1,000 containers scheduled because that number was faulty. So that's something that we're thinking about. And there's a couple other things like, a couple of things that we don't do, like privileged containers expect to interact directly with node resources. We don't allow that with VK. You cannot directly talk to the node. You can't directly access any of the node files or anything like that. So some monitoring things don't work exactly as expected. So yeah, there's just a couple of things that you have to think about. And then a node is a single fault domain. It's not true in VK. The fault domain is actually probably the container, but how do we expose that? Because the providers get to choose where the fault domains lie. So there's just some questions that we've always had. Do you want to take it? Yeah. You got it. So going off of what Rio was saying, a lot of the work that we're doing in the virtual Kubler project right now is to be more compatible with the full Kubernetes API, whether that's obviously being able to create, delete pods, gather metrics from pods, attach volumes, support GPU type of pods, et cetera. You name it. Yeah. You want to add me? Cool. All right, so virtual Kublet. That was kind of what the node does and I tried to give a little bit of an explanation of what VK did layered on top of it. The virtual Kublet itself, we treat the concept of pods and nodes in the abstract. So we're literally just passing through a string when you see a node, a virtual Kublet node. It's just a string that's being passed through to Kubernetes. A node is about a resource again, but the way that we've set it up, it isn't technically. So the specifics of how you create a pod, delete a pod, update a pod, add files, like volumes for example, things like that or like config maps and secrets and all of that are all contained within the interface itself, but the logic for how all of that happens is your service API. So all of the provider APIs get connected up to the Kubernetes APIs through that interface. The common code in virtual Kublet is relatively small because it's very provider-specific code. We basically are a home for all of these providers to interact together, but the logic for how things work are held within the provider repos itself. One interesting one is basically service discovery, because how do you do that when you just have a concept of like a node poppy in Kubernetes? You need a Kube proxy basically on the node to be able to talk out to the world and have service discovery. In Azure, I know what we've done is we spin up a Kube proxy as a sidecar to every container instance that spins up, but the way that maybe Alibaba does that or Huawei does that might be different. I actually think they do a very similar thing, but that's completely abstracted away from the user. In Kubernetes today in managed service, you would actually see that Kube proxy and that Kublet. For this, you just see this node with no Kublet and no Kube proxy and none of that. It's all abstracted away, which is kind of nice if it works. Sure. Yes. It means the one Kubernetes can have more Kubernetes because one Kublet is a node in the KBus. If it has a pod, it means that Kubernetes can be more Kubernetes. Yeah, so you can have a bigger Kubernetes cluster with this one node that you can layer multiple pods, XMeta pods that isn't bounded by a node resource. I think that answers your question, hopefully. You could layer Kubernetes in Kubernetes if that was another question you were asking, if you can have virtual Kublets spin out to another Kubernetes cluster that's completely possible because it's just an API that you're calling out to. Well, we'll show an architecture diagram of what a cluster would look like with a virtual node on it in a bit. Yeah. Okay, YVK, we've basically abstracted away what it means to be a Kublet in Kubernetes and that's actually really interesting to a lot of people. For example, if you're trying to mask another orchestrator and you want to use the Kubernetes APIs, but you don't want to lose everything you've built with another orchestrator. So if you go check out our repo right now, there's some interesting things happening from other providers that are trying to do that. Yeah, so a couple of use cases. We have the one I talked about before for fast bursting. If you have a sudden amount of load, basically, for example, if you're a meter reading company and you need to scale up the amount of compute resources that you have depending on if there's a natural disaster. So if there's a natural disaster, you need to now read all of the meters to understand how much electricity is there and what's gone down and what you need to go and fix. In that use case, virtual Kublet's really useful because you weren't planning for that, so you didn't have to have that capacity already there for you. You can just automatically spin out into VK and the provider itself would have hopefully abstracted away the amount of compute resources. So you would have this unbounded amount of compute resources. And then we have some IoT Edge use cases and then also masking virtual Kublet as a different kind of node. So you could mask it as a GPU node or a GPU container or things like that or a different SKU. It's all magic. Do you want to do this part? Okay, sure. So this is what, if you want to be a provider, this is what you end up doing. You end up filling in this code. If you can understand how to create a pod and delete a pod and update, well, update is a little bit questionable because no one's really done that yet, but basically delete a pod and get a pod and possibly get logs. You can be a provider in virtual Kublet. It's a really low bar. So if you're dominoes and you want to, if you create a pod and you want to get a pizza, when you get that pod, you can do it. I don't know if I explained that very well, but anyone can be a provider, so please join us. And this is really what you have to do. You just have to provide the back end plumbing necessary and support the life cycle management of pods and Kubernetes. So that's all your logic. It's not something we do, but all of the code is open source so you can copy and paste from any of the other providers or look at it. So extend Kubernetes to anything that you want. I've heard of some interesting, a couple other use cases that I've heard of is they want to mask, someone wanted to mask a satellite that was orbiting Earth as a node in Kubernetes. So you can think, now with virtual kubelet you can use Kubernetes and the Kubernetes API as a common platform for any developer or any infrastructure operator and you can mask other compute resources if you so choose, which is really interesting. So you can mask dominoes. That's an example I really like, because I love pizza, but for example, if you created a pod, it would spin out to dominoes and order you pizza, but you have to create that logic. But yeah, a lot of people are using it for the fast burst use case where they just spin up a bunch of containers really quickly, but you can also mask maybe IoT edge use cases. So you could have a node representing a device. So you could have a node representing a fridge and any time you spin out a pod to that fridge, it would maybe update the fridges software or something or maybe you would add a new feature to it. So we're really just trying to make Kubernetes the basic common control plan for any sort of workload and any sort of device, hopefully in the world. Yeah, and here's some examples of the providers that have already implemented the virtual kubla interface, Alibaba, Azure, AWS, just to name a few big ones. This is a sampled architecture diagram of what it would look like with an IoT edge deployment. So as Rhea was saying, we want the user to just, who's already familiar with Kubernetes, not have to learn anything new. You can manage all software and containers on your edge devices just by using the IoT edge provider with your already existing Kubernetes cluster. You don't have to worry about nodes or what or anything like that. You can just spawn off workloads and containers to those edge devices. I saw yesterday that there was a cool lightning talk of the guy that built a Kubernetes cluster with just a bunch of raspberry pies and using tape in the rubber band together. So he has his own Kubernetes cluster and you can use that for testing, for example. Yeah. So this is what it looks like with Azure. So in Azure, we have a service called Azure Kubernetes Service. It's a managed Kubernetes offering where Azure takes care of managing all of the master nodes. So you as a customer only have to worry about the worker nodes and the workloads that are on them. But even as you have to manage and own a plan for capacity on the worker nodes, this can get very expensive. Worker nodes can be underutilized and the startup time to add more worker nodes if you need to scale on demand pretty quickly can be very slow. So we've introduced an offering called Azure Virtual, or AKS Virtual Nodes which uses the Virtual ClubeLit project and now you can burst out capacity of your pods to Azure Container Instances. Azure Container Instances is our Azure's serverless containers offering where a user can just send deployed containers and we manage all the infrastructure. So you as a user only have to worry about developing your application packaging into a container and giving it to Azure. Yes. Yeah, this is a generally available offering. So there's two products. There's Azure Kubernetes Service and Azure Container Instances. If you're using AKS, then there's a feature you can enable to turn on this Virtual Nodes experience. Yeah, yes. Currently it's not available with that, no. But that's on our roadmap. So since this is an open source project you can pave it onto any Kubernetes cluster. So it doesn't matter if we offer it. We don't support it. I mean I guess we support it now in Azure but with OpenShift we wouldn't exactly support it but as long as you have access to your VMs you can deploy Virtual ClubeLit and it will spin out to ACI. You have to fill in the details yourself. So go to our Virtual ClubeLit repo itself for those instructions. So yes. Yeah, that's a really good point. The way Azure provides here is we do a lot of that setup for you and configure a lot of networking and do some other plumbing to make the experience of installing VK and deploying pods to ACI more seamless. But yeah, theoretically any Kubernetes cluster even if it's OpenShift you can install Virtual ClubeLit and burst out to ACI or any one of the other providers. Yeah, so an example use case of this is there's a company in Africa. So they're a bank and whenever they do a marketing campaign for loans or whatever or credit cards they see a spike in traffic to their services. So instead of having, let's say they run a three node cluster right now instead of having five nodes and two of those nodes being very underutilized and then paying for them 24 seven now they just can take their three node cluster and then spin off extra pods into ACI. Whenever that marketing campaign is done maybe it's over a weekend or something they can spin that back down. Other examples are in the retail industry so during Black Friday or Singles Day for example you're gonna see maybe two times, three times maybe even more as much traffic in. You don't wanna be provisioning infrastructure that's gonna be underutilized. So yeah, and just to wrap this up this diagram shows you that some of the virtual machines there's gonna be empty space that's wasted but if you're using virtual nodes you can just spin out the pods to Azure Container Instances and then create and destroy them as you wish. One thing I didn't mention is also this is a lot faster in terms of speed because you don't have to spin up new virtual machines to put the pods on them. Azure Container Instances we already have virtual machines that are running hot that are abstracted away from users and the pods will just get placed on those. So this is a part of the pod spec that you would need to include when using virtual kubelet and I'll show a full YAML file in a bit but what this shows is on the virtual node we add what's called a taint in Kubernetes. You can think of it as kind of like a lock and if you add tolerations to your pod spec this is basically the key to be able to provision workloads onto the node with that lock. The reason why we do this is as Rhea was saying the full compatibility of the Kubernetes API there's not everything is available with virtual kubelet so some special types of workloads might not work the way that they're expected and if you use Azure Container Instances for example. Yeah, yes. One of the things I think you didn't explain well is that why the virtual kubelet can actually satisfy this kind of bursting kind of scenario why it can launch the ports so quickly in the normal way I think, so yeah, that's actually my question. Is that because of the provider? Yeah, the providers are the ones that have the infrastructure in which the pods are placed. So typically speaking and feel free to add more many of all the providers or many of the providers are the each company's serverless container offerings like Fargate for example or ACI. So since the nodes are already exist all that need the work that actually happens is the container image is downloaded to that node and then the container is just started up rather than having to boot up a full OS and then install the image and start to container. I gave that, I think that the differentiator comes from the provider itself rather than from the virtual kubelet. It can, but there's a couple as a vendor neutral person I'm gonna step back from being Microsoft right now. This is a really good solution for Microsoft because RVMs might be slower to boot up. If you look at other providers they might be as fast as containers are today and then this use case doesn't exactly work for everyone. Another thing, I think another reason for why this project, a lot of people like it is the fact that it abstracts away the kubelet so much. So past the first use case it's really useful for Microsoft but it's not super useful for some other companies. Another thing is customers don't wanna have to CVMs or manage them. So in that standpoint that's why like Amazon and Alibaba have implemented this not just because maybe their infrastructure is already fast but because they don't want their customers to have to think about nodes anymore. You just think about containers so you just see pots. That's all you care about. So if a node goes down you actually don't know and you don't care about that. That's probably the vendor neutral. Hello. So I see this is a really good use case but I'm just worried like how does or rather I have a question that how does the underlying things like the network and like how does even Azure in AKS how does it address those things? Like how do I enforce to which network it goes to or the NSGs? We have a concept of NSGs in Azure, right? How does that enforce that? Where goes the security? Do we put network policies or how do we do it? Yeah, so that's one way you can do that. With Azure Container Instances the Azure networking team has done a lot of special work in order for the pods that are deployed to ACI to be contained within the same virtual network as the full AKS cluster. So any features that you can get with Azure virtual networks whether it's NSGs or custom routing, DNS, et cetera that's all provided. Granted, the full parity isn't there if you're just using a virtual network with AKS but that's something that we're working to reach. So you mean that when I deploy the AKS and I choose a network, whatever network segmentation we need and all those I choose basically the pods get spin up into the same thing, right? And it will follow whatever the network we specify at the beginning. Yeah, so anything that you, any Azure virtual network features that you would use or apply to if you're just using a single AKS cluster you can do the same if you're using the AKS cluster and a virtual nodes in ACI. Thanks. No, so VK has a number of other providers. So I know we're gonna talk about another one as well from Ali Baba, is there a question? So yeah, when you install a virtual kubelet onto your Kubernetes cluster you specify which provider you want to use. ACI is one of them. And in Azure when you're using, it's just one. Yeah, there are many different ones. Yeah. I didn't see actually So does the provider have to like implement all the details and technical details about network, you know, storage kind of stuff just mentioned? That is explicit. That's something that we chose to not implement because the network is very provider specific, right? If you go to Ali Baba's cloud or you go to Amazon's cloud or you go to Google's cloud the network policies and the way that people create networks is very specific to a cloud provider. We don't have networks that span Google and Microsoft yet, right? You can't create a network for a cluster that's in Azure and have it span there. So it's provider specific. You have to figure it out yourself from our perspective. Yeah, okay. Can I get my hardware information from a pod of ACI like my CPU type and my disk type? Not that from the node itself. So the node, you can still see the node resource there and when you basically describe that node you'll still be able to see what OS type it is, how much compute resources you have and how much disk space you have. I would just be wary about that because they're fake numbers. They're numbers that the provider chooses to pass through. For the OS type, it's the same thing. Like we'll make it easier for you to spin out to that specific OS type if we say Windows and you deploy a pod to that node. It'll span out to Windows but sometimes you can add different flags in your pod spec itself and actually spin out to Linux on that Windows node. It's like super abstract basically. But if I care about the performance how can I compare the performance of my true node and my virtual node? So when you deploy a pod to the virtual node it's just like any other pod used but you can specify how much CPU and memory that'll be in that pod and you can use the kubectl commands to see how much is actually being utilized. So that information is returned from ACI but at a pod level. Yes, I think you are saying the resource but what I care about is the like my CPU frequencies and my disk IO like this information because in the AKS I can see the node I can choose the node type like A2 or D type but in the virtual node I cannot see this information. Since these pods are living on ACI at least in the ACI case we emit a number of metrics out which can be consumed either within the cluster itself or within Azure monitor offerings. So that includes like network IO, memory, CPU, a few others, disk I believe as well. So that's one way you can read that, yeah. Okay, so can I understand that in ACI you can make sure that every pod can provide the same performance per CPU and memory, right? Yeah. Performance is iffy. You have to figure that out kind of. We don't have an SLA. So and real quick, another piece of code that you can add to your YAMLs is with Affinity. So using this you can choose certain pods to go on to the virtual node. The thing I showed before is just the key in order to deploy on to the node with the lock. If you want a specific pod to get placed on the virtual node you can add this Affinity. You can either do the preferred during scheduling or required during scheduling and yeah. One thing I wanna show you guys I can pull it up very quickly. Anyone have a question while we're, yeah. In normal Kubernetes cluster, we have monitor like Prometheus and they can fetch metrics from Qubelet that the metrics about the node and about the pods and all work load on that. But on Qubelet I did interface that there was nodes such an interface that exposed the metrics to the monitor components. So if I choose to use a component and specify provider. So I can only depend on the monitors that provided by the provider or I can still integrate it with Prometheus or other Kubernetes monitor system. Yeah, so I mean I can speak for ACI. We make sure that the metrics that are emitted are written back to the metric server of the Kubernetes cluster. So you can use Prometheus and Grafana and actually read CPU and memory. You can also write custom metrics such as like requests per second that are being tracked by your ACI pods to the metric server and then that can be all read through a familiar monitoring solution like Prometheus. Okay, so it's to provide us responsibility to handle the metrics and where to write to and then. So there's not a standard interface for that. We haven't made a standard for any of that. For example, we're at V1 right now but as we get to maybe V2, maybe these are standards we should make in Virtual Qubelet for providers because today only the interface is production ready. We haven't, Virtual Qubelet the project hasn't said that any of the providers are production ready because we have no standards around those. So like Azure has gone out on their own and said that they're GA but Virtual Qubelet the project does not have an interaction with Azure to make sure that they're actually ready to be GA. And I think that's a set of standards that we're gonna start creating in the next year. You're completely correct. Like we probably should have some sort of standard for that. Okay, go ahead. Just to wrap up and then we can take more questions at the end. This is a sample YAML file that you can use with Virtual Qubelet. A lot of this should look familiar to you. You can use images as normal. I use an image that's in Azure Container Registry. You can read from Kubernetes secrets. And here I use the node selector, Placement Constraint instead of Affinity and then my toleration is here as well. And then I'm gonna hand it back to you. Dolly Bobble. Okay, sure. All right, because I think we're out of time. So we can take a couple more questions. It's 35 minutes, right? Yeah, yeah, we're time out. Okay. Yeah, we can take some questions up here if you guys wanna come up or stick around. But thank you so much for listening. It's good to see that a lot of people are still excited about this project. And I mean, we're gardening more and more popularity as time goes on, which is really awesome. So come join us. We're a really open community. We have a Slack channel on Kubernetes. And yeah, we'd love to talk to you. So thank you. Oh, yeah, virtual kubelet. It's just virtual dash kubelet. That's the organization. Yeah, feel free to connect with us over GitHub and Twitter. And yeah, I don't have... Oh, the internet's working now. Oh, nice. Oh. Oh man. All right. Thanks guys. Thank you.